Maximum Entropy Solution Research Articles

Metastability and rapid convergence to quasi-stationary bar states for the two-dimensional Navier–Stokes equations

Quasi-stationary, or metastable, states play an important role in two-dimensional turbulent fluid flows, where they often emerge on timescales much shorter than the viscous timescale, and then dominate the dynamics for very long time intervals. In this paper we propose a dynamical systems explanation of the metastability of an explicit family of solutions, referred to as bar states, of the two-dimensional incompressible Navier–Stokes equation on the torus. These states are physically relevant because they are associated with certain maximum entropy solutions of the Euler equations, and they have been observed as one type of metastable state in numerical studies of two-dimensional turbulence. For small viscosity (high Reynolds number), these states are quasi-stationary in the sense that they decay on the slow, viscous timescale. Linearization about these states leads to a time-dependent operator. We show that if we approximate this operator by dropping a higher-order, non-local term, it produces a decay rate much faster than the viscous decay rate. We also provide numerical evidence that the same result holds for the full linear operator, and that our theoretical results give the optimal decay rate in this setting.

A derivation of the master equation from path entropy maximization

The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive nth-order Markov processes and the master equation as unique solutions to an inverse problem. We find that when constraints are not enough to uniquely determine the stochastic model, an nth-order Markov process emerges as the unique maximum entropy solution to this otherwise underdetermined problem. This gives a rigorous alternative for justifying such models while providing a systematic recipe for generalizing widely accepted stochastic models usually assumed to follow from the first principles.

Approximate maximum entropy on the mean for instrumental variable regression

We want to estimate an unknown finite measure μX from a noisy observation of generalized moments of μX, defined as the integral of a continuous function Φ with respect to μX. Assuming that only a quadratic approximation Φm is available, we define an approximate maximum entropy solution as a minimizer of a convex functional subject to a sequence of convex constraints. We establish asymptotic properties of the approximate solution under regularity assumptions on the convex functional, and we study an application of this result to instrumental variable estimation.

The SIS and SIR stochastic epidemic models: A maximum entropy approach

We analyze the dynamics of infectious disease spread by formulating the maximum entropy (ME) solutions of the susceptible-infected-susceptible (SIS) and the susceptible-infected-removed (SIR) stochastic models. Several scenarios providing helpful insight into the use of the ME formalism for epidemic modeling are identified. The ME results are illustrated with respect to several descriptors, including the number of recovered individuals and the time to extinction. An application to infectious data from outbreaks of extended spectrum beta lactamase (ESBL) in a hospital is also considered.

Hausdorff moment problem and Maximum Entropy: On the existence conditions

Different existence conditions of the Maximum Entropy solution to finite Hausdorff moment problem have been formulated in literature. Through a counterexample we prove that the most cited one is uncorrect. We do not bound ourselves to a crude counterexample, as we think that a detailed explanation is of interest by itself. It clarifies the difference existing between the finite and infinite Hausdorff moment problem existence conditions.

A Maximum Entropy Solution of the Covariance Extension Problem for Reciprocal Processes

Stationary reciprocal processes defined on a finite interval of the integer line can be seen as a special class of Markov random fields restricted to one dimension. Nonstationary reciprocal processes have been extensively studied in the past especially by Jamison et al. The specialization of the nonstationary theory to the stationary case, however, does not seem to have been pursued in sufficient depth in the literature. Stationary reciprocal processes (and reciprocal stochastic models) are potentially useful for describing signals which naturally live in a finite region of the time (or space) line. Estimation or identification of these models starting from observed data seems still to be an open problem which can lead to many interesting applications in signal and image processing. In this paper, we discuss a class of reciprocal processes which is the acausal analog of auto-regressive (AR) processes, familiar in control and signal processing. We show that maximum likelihood identification of these processes leads to a covariance extension problem for block-circulant covariance matrices. This generalizes the famous covariance band extension problem for stationary processes on the integer line. As in the usual stationary setting on the integer line, the covariance extension problem turns out to be a basic conceptual and practical step in solving the identification problem. We show that the maximum entropy principle leads to a complete solution of the problem.

Phase Coupling Estimation from Multivariate Phase Statistics

Coupled oscillators are prevalent throughout the physical world. Dynamical system formulations of weakly coupled oscillator systems have proven effective at capturing the properties of real-world systems and are compelling models of neural systems. However, these formulations usually deal with the forward problem: simulating a system from known coupling parameters. Here we provide a solution to the inverse problem: determining the coupling parameters from measurements. Starting from the dynamic equations of a system of symmetrically coupled phase oscillators, given by a nonlinear Langevin equation, we derive the corresponding equilibrium distribution. This formulation leads us to the maximum entropy distribution that captures pairwise phase relationships. To solve the inverse problem for this distribution, we derive a closed-form solution for estimating the phase coupling parameters from observed phase statistics. Through simulations, we show that the algorithm performs well in high dimensions (d = 100) and in cases with limited data (as few as 100 samples per dimension). In addition, we derive a regularized solution to the estimation and show that the resulting procedure improves performance when only a limited amount of data is available. Because the distribution serves as the unique maximum entropy solution for pairwise phase statistics, phase coupling estimation can be broadly applied in any situation where phase measurements are made. Under the physical interpretation, the model may be used for inferring coupling relationships within cortical networks.

Function reconstruction as a classical moment problem: a maximum entropy approach

We present a systematic study of the reconstruction of non-negative functions via maximum entropy approach utilizing the information contained in a finite number of moments of the functions. For testing the efficacy of the approach, we reconstruct a set of functions using an iterative entropy optimization scheme, and study the convergence profile as the number of moments is increased. A wide variety of functions are considered that include a distribution with a sharp discontinuity, an oscillatory function, a distribution with singularities and finally a distribution with several spikes and fine structure. The last example is important in the context of the determination of the natural density of the logistic map. The convergence of the method is studied by comparing the moments of the approximated functions with the exact ones. Furthermore, by varying the number of moments and iterations, we examine to what extent the features of the functions, such as the divergence behavior at singular points within the interval, is reproduced. The proximity of the reconstructed maximum entropy solution to the exact solution is examined via Kullback–Leibler divergence and variation measures for different number of moments.

Chordal Graphs to Identify Graphical Model Solutions of Maximum of Entropy Under Constraints on Marginals

We consider the problem of specifying the joint distribution of a collection of variables with maximum entropy when a set of marginals are fixed. One can easily derive that the structure of the solution joint distribution is that of a graphical model. The potential functions are then marginals at some power. We address the following question, Under which conditions on the set of constraints is it possible to fully identify the canonical exponents in the maximum entropy solution as functions of the problem structure? Literature related to this topic is somewhat scattered in disciplines such as statistical mechanics, information theory, graph theory, and inference in graphical models. In this article we gather and link results from these different fields. From this, we show that for a particular class of constraints set on marginals, the chordal maximal coherent sets of constraints, it is possible to derive the canonical exponents of the graphical model solution of the maximum entropy problem as the numbers of occurrences of separators in an associated join tree. Conversely, we present sufficient conditions to ensure that a graphical model is a solution of a maximum entropy problem.

Deconvolution with Maximum Entropy Solution to Determine Local Extinction Coefficient and Local Volume Concentration Values from Laser Diffraction Data

AbstractThe traditional use of the laser diffraction technique provides line‐of‐sight liquid spray drop‐size distribution. However, deconvolution of the measurements can be performed for axisymmetric spray in order to determine local spray characteristics. In a previous publication, a new deconvolution technique making use of the maximum entropy principle was established and applied to determine the local drop‐size distributions. The entire approach was experimentally validated. In this work, the technique is employed to determine local extinction coefficient values. As in the previous investigation, the measurement procedure consists of scanning a laser beam through the spray cross‐section from the center to the edge of the spray. By use of the transmittance theory, the local extinction coefficients allow the local volume concentrations to be calculated. This theory introduces the mean scattering coefficient. The results show that this coefficient must be determined as a function of the Sauter mean diameter in order to avoid overestimation of the volume concentration. Although no proper validation is presented, the coherence of the overall approach is discussed in detail and solutions for improving the spatial resolution are presented. Finally, the local volume concentrations are combined with the local drop‐size distribution to provide local volume‐weighted, drop‐size distributions. These distributions provide information on the localization of the drops according to their diameter as well as on the spatial liquid distribution. This work illustrates applications and performances of laser diffraction technique that are rarely used.

Iterative maximum entropy power spectrum estimation for multirate systems

In multirate systems, observations are generally insufficient to determine the power spectrum of the input signal. In this paper, we reformulate the problem using a novel matrix notation and the discrete entropy function. Then we present an iterative maximum entropy power spectrum estimation algorithm for the solution of this problem. Contrary to the existing solutions, the new algorithm is computationally efficient since it is based on fast Fourier transform (FFT) and simple matrix calculations. Furthermore, simulation results show that the new algorithm converges to the maximum entropy solution and can be successfully used in multirate statistical data estimation.

Performance analysis of a threshold-based discrete-time queue using maximum entropy

The provision of guaranteed QoS for various Internet traffic types has become a challenging problem for researchers. New Internet applications, mostly multimedia-based, require differentiated treatments under certain QoS constraints. Due to a rapid increase in these new services, Internet routers are facing serious traffic congestion problems. This paper presents an approximate analytical performance model in a discrete-time queue, based on closed form expressions using queue threshold, to control the congestion caused by the bursty Internet traffic. The methodology of maximum entropy (ME) has been used to characterize closed form expressions for the state and blocking probabilities. A discrete-time GGeo/GGeo/1/{ N 1, N 2} censored queue with finite capacity, N 2, external compound Bernoulli traffic process and generalised geometric transmission times under a first come first serve (FCFS) rule and arrival first (AF) buffer management policy has been used for the solution process. To satisfy the low delay along with high throughput, a threshold, N 1, has been incorporated to slow the arrival process from mean arrival rate λ 1 to λ 2 once the instantaneous queue length has been reached, otherwise the source operates normally. This creates an implicit feedback from the queue to the arrival process. The system can be potentially used as a model for congestion control based on the Random Early Detection (RED) mechanism. Typical numerical experiments have been included to show the credibility of ME solution against simulation for various performance measures and to demonstrate the performance evaluation of the proposed analytical model.

Tsallis distribution as a standard maximum entropy solution with ‘tail’ constraint

We show that Tsallis' distributions can be derived from the standard (Shannon) maximum entropy setting, by incorporating a constraint on the divergence between the distribution and another distribution imagined as its tail. In this setting, we find an underlying entropy which is the Rényi entropy. Furthermore, escort distributions and generalized means appear as a direct consequence of the construction. Finally, the “maximum entropy tail distribution” is identified as a Generalized Pareto Distribution.

Maximum entropy solution to ill-posed inverse problems with approximately known operator

We consider the linear inverse problem of reconstructing an unknown finite measure μ from a noisy observation of a generalized moment of μ defined as the integral of a continuous and bounded operator Φ with respect to μ. Motivated by various applications, we focus on the case where the operator Φ is unknown; instead, only an approximation Φ m to it is available. An approximate maximum entropy solution to the inverse problem is introduced in the form of a minimizer of a convex functional subject to a sequence of convex constraints. Under several assumptions on the convex functional, the convergence of the approximate solution is established.

A maximum entropy principle based closure method for macro-micro models of polymeric materials

We consider the finite extensible nonlinear elasticity (FENE) dumbbell model in viscoelastic polymeric fluids. We employ the maximum entropy principle for FENE model to obtain the solution which maximizes the entropy of FENE model in stationary situations. Then we approximate the maximum entropy solution using the second order terms in microscopic configuration field to get an probability density function (PDF). The approximated PDF gives a solution to avoid the difficulties caused by the nonlinearity of FENE model. We perform the moment-closure approximation procedure with the PDF approximated from the maximum entropy solution, and compute the induced macroscopic stresses. We also show that the moment-closure system satisfies the energy dissipation law. Finally, we show some numerical simulations to verify the PDF and moment-closure system.

Passivity-preserving model reduction by analytic interpolation

Antoulas and Sorensen have recently proposed a passivity-preserving model-reduction method of linear systems based on Krylov projections. The idea is to approximate a positive-real rational transfer function with one of lower degree. The method is based on an observation by Antoulas (in the single-input/single-output case) that if the approximant is preserving a subset of the spectral zeros and takes the same values as the original transfer function in the mirror points of the preserved spectral zeros, then the approximant is also positive real. However, this turns out to be a special solution in the theory of analytic interpolation with degree constraint developed by Byrnes, Georgiou and Lindquist, namely the maximum-entropy (central) solution. By tuning the interpolation points and the spectral zeros, as prescribed by this theory, one is able to obtain considerably better reduced-order models. We also show that, in the multi-input/multi-output case, Sorensen’s algorithm actually amounts to tangential Nevanlinna–Pick interpolation.

Maximum entropy solutions for batch arrival queue with an un-reliable server and delaying vacations

We consider the M [ x] / G/1 queueing system with server vacations characterized by a special feature, reflecting real-life situations, in which the server, upon finding an empty system at the end of a vacation, activates a timer of duration T and waits dormant in the system. If a batch arrives during the dormant period, the server starts providing his service for the waiting customers, but if no arrivals occur, the server waits no more and takes another vacation. The server is subject to breakdowns according to a Poisson process and his repair time obeys an arbitrary distribution. We use maximum entropy principle to derive the approximate formulas for the steady-state probability distributions of the queue length. We perform a comparative analysis between the approximate results with established exact results for various vacation time, timer duration, service time and repair time distributions. We demonstrate that the maximum entropy approach is accurate enough for practical purpose and is a useful method for solving complex queueing systems.

Maximum entropy principle and classical evolution equations with source terms

We devise a maximum entropy technique to construct (approximate) time-dependent solutions to evolution equations endowed with source terms and, consequently, not preserving normalization. In some special cases the method yields exact solutions. It is shown that the present implementation of the maximum entropy prescription always (even in the case of approximate solutions) preserves the exact functional relationship between the time derivative of the entropy and the time-dependent solutions of the evolution equation. Other properties of the maximum entropy solutions and some illustrative examples are also discussed.

Performance of Queueing Model of one UMTS Cell with R Virtual Zones by the Maximum Entropy Solution

UMTS stands for Universal Mobile Telecommunications System, which is one of the third generation (i.e., 3G) cellular systems, will come into full commercial phase by year 2005 and the first UMTS services are launched commercially in 2001. Therefore, the research of UMTS is an important and urgent task. In this paper, the performance of UMTS systems is studied through a queueing model which consists of one UMTS with many virtual zones to evaluate the system uplink traffic performance. The two performance measures: blocking probability and system utilization are obtained by the Maximum Entropy Principle, i.e., ME solutions for S RXR (GGeo/GGeo/1)/N queue. S RXR (GGeo/GGeo/1)/N system means that arrival time and service time are both GGeo (Generalized Geometric) distributed, each output port has a single server and the maximum capacity is N. After the theoretical analysis, the numerical results are found for the idiographic example, 8×8 (GGeo/GGeo/1)/512 queueing model. Additionally, the performance of this queueing model is discussed with the effect of the parameters Ca 2 (squared coefficient of variation of the interarrival time), Cs 2 (squared coefficient of variation of the service time) and the arrival rate, ?.

Maximum entropy approach to the collisional Vlasov equation: Exact solutions

We show that the maximum entropy approach proposed by El-Wakil, Elhanbaly, and Abdou (from now on EEA) in [S.A. El-Wakil, A. Elhanbaly, M.A. Abdou, Physica A 323 (2003) 213] for solving approximately the collisional Vlasov equation actually can provide exact solutions if properly implemented. We consider here two alternative procedures for obtaining exact maximum entropy solutions of the aforementioned equation. On the one hand, after identifying an appropriate set of relevant mean values (moments), we show that there are exact maximum entropy solutions associated with that set of moments. These solutions can be studied focusing either on the equations of motion of the moments themselves, or on the equations of motion of the corresponding Lagrange multipliers. On the other hand, it is possible to find exact solutions of the reduced equation considered by EEA, if one takes explicitly into account the zeroth-order moment of the solutions.