Minimum Entropy Method Research Articles

Minimum relative entropy: Theory and application to surface temperature reconstruction from borehole temperature measurements

In this paper we extend the minimum relative entropy (MRE) method to reconstruct ground surface temperature changes (GST) from borehole temperature measurements (BHT). The application of MRE to recovering GST is promising and provides an alternative to other inverse methods in geophysics. The relative entropy formulation provides the advantage of allowing for a prior bias in the estimated pdf and ‘hard’ bounds if desired. Test cases showed good recoveries of the GST. The method was utilized in recovering GST from two data sets in Canada. The Lac Dufault data gave very consistent results for different choices of a priori information and bounds. The Mariner results were not as good quality. This method has only recovered past ground surface temperatures, which does not directly provide information regarding climate change. However, results contained herein show fairly uniform temperatures until the past 100 to 500 years and in more recent times about a 4°C rise in temperature, consistent with previous published results.

Novel approaches to entropic regularization

In this work, two new entropic regularization techniques are introduced. They represent a generalization of the standard MaxEnt regularization method, and allow for a greater flexibility for introducing any prior information about the expected structure of the true physical model, or its derivatives, into the inversion procedure. The first technique is based on the minimization of the entropy of the vector of first-differences of unknown parameters. Adopting standard terminology, it is known as the minimum first-order entropy method (MinEnt-1). To illustrate the essential feature of the method, MinEnt-1 is applied to the reconstruction of two-dimensional geoelectric conductivity distributions from magnetotelluric data. The second technique is based on the maximization of the entropy of the vector of second-differences of the unknown parameters, and is denoted as the MaxEnt-2 method. The MaxEnt-2 method is applied to the retrieval of vertical profiles of temperature in the atmosphere from remote sensing data.

Astronomical time-series analysis — III. The role of the observational errors in the minimum entropy method

In two recent papers a new method for searching for periodicity in time series was introduced. It takes advantage of the Shannon entropy to compute the amount of information contained in the light curve of a given signal as a function of a supposed period p. The basic result is that, if the signal is T-periodic, the entropy is then minimum when pT. Also, there is theoretical and numerical evidence that the minimum entropy method is more sensitive to the presence of periodicity and has a higher resolution power than other classical techniques. In the present work the discussion is focused on the way in which the observational errors have to be included in the method. The application of the resulting modified algorithm to real data and a performance comparison with the former algorithm are presented. The dependence of both periodograms on the size of the partition is also investigated. Analytical estimates are given only for the limiting case of small errors. The numerical results show that the new algorithm leads to a smoother periodogram and provides a higher significance for the minimum than the former algorithm.

Analyzing Characters of Plasma Wake with Method of Chaos Dynamics

In this paper, analyzing the experiment data of multiple scanning probes in the arc wind-driven hypersonic plasma flows with the chaos dynamics method, we have obtained the power spectrum, the frequency wavenumber spectral density, the correlation dimension and Kolmogorov entropy of the flows, respectively. The distribution of correlation dimension and Kolmogorov entropy of the blunt cone wakes along the axis and the radius of the flows are presented successfully in the flows. The distributions clearly show the structure of turbulence and lamination in blunt cone wakes. Using the method of minimum Kolmogorov entropy, we observed the structure of the bow shock at far wake region in the blunt cone wakes.

Three-dimensional plume source reconstruction using minimum relative entropy inversion

In this paper we extend the minimum relative entropy (MRE) method to recover the source-release history of a three dimensional plume. This extension is carried out in an analytic framework, and in order to qualify as a linear inverse problem the various transport parameters such as dispersivity and the like are considered to be known. In addition, the groundwater flow system is assumed to be steady and uniform. The contributions of this paper include an explanation how MRE can be used as a measure of resolution in linear inversion, a reporting of a three dimensional analytic solution for mass transport in a steady one dimensional velocity field for a variable in-time source loading, an estimation the source-release history for synthetically generated data sets, and an application of the methodology to a case-study problem at the Gloucester Landfill in Ontario, Canada. We found that the relative entropy measure is useful in indicating the reduction in uncertainty between the posterior and prior pdfs as a result of the new information provided by the physical constraints and data. Using the individual model-parameter relative entropies as a measure of resolution, one can make quantitative judgments about which part of the history is likely to be well resolved. Comparing inversion results for synthetic aquifers with one well and two sample points with only one sample point indicates that temporal data at several wells allows for a superior reconstruction of the release history. We investigate the potential benefits of locating sample points on an spatial rather than temporal basis. Results show that early part of the release history is poorly recovered. Comparing these results with one well and two sample points indicates that temporal data at a few wells allows for a better reconstruction of the release history. An incomplete time record is also investigated. Results show that early part of the release history prior to the commencement of measurements is poorly recovered. It is essential that as much as possible of the time histories of plumes be monitored if the entire release history is to be determined. The MRE approach is used to reconstruct the release history of a 1,4-dioxane plume measured at the Gloucester Landfill in Ontario, Canada. The recovered release history is fairly narrow and generally flat in shape, although two peaks are evident. Neither peak is particularly well defined, judging from the resolution curve and analysis of the confidence ranges. One of the peaks coincides with year 1979, close to the year 1978 in which a large spill was noted. However, a considerable variation of possible source releases is possible.

A Minimum-Entropy Algorithm for Surface Phasing Problems

Methods of determining the phases for noisy and incomplete surface diffraction data (X-ray or transmission electron) are investigated. It is found that, while conventional methods do not always work, a minimum-entropy method that uses the relative entropy is more effective in finding the correct solutions.

Numerical inversion of two-dimensional geoelectric conductivity distributions from magnetotelluric data

In this paper, a new inversion technique, called the minimum first-order entropy (MinEnt-1) method, is proposed for the reconstruction of two-dimensional geoelectric conductivity distributions from magnetotelluric (MT) data. The method combines an iterative search with a regularization technique based on the minimization of the entropy measure of the vector of first-differences of the unknown conductivities. Numerical simulations, using synthetic data corrupted with gaussian noise, show that the MinEnt-1 algorithm converges to excellent conductivity reconstructions, yielding in many cases results that are superior to those obtained by the maximum entropy formalism. Unlike other classical regularization schemes, which maximize smoothness for a given data, the proposed method constrains the class of possible solutions into a restricted set of low entropy models, constituted by locally smooth regions separated by sharp discontinuities. This may be an effective approach for the incorporation of prior information about the local smoothness of the real physical model.

Pharmacokinetic parameter estimations by minimum relative entropy method

For estimating pharmacokinetic parameters, we introduce the minimum relative entropy (MRE) method and compare its performance with least squares methods. There are several variants of least squares, such as ordinary least squares (OLS), weighted least squares, and iteratively reweighted least squares. In addition to these traditional methods, even extended least squares (ELS), a relatively new approach to nonlinear regression analysis, can be regarded as a variant of least squares. These methods are different from each other in their manner of handling weights. It has been recognized that least squares methods with an inadequate weighting scheme may cause misleading results (the "choice of weights" problem). Although least squares with uniform weights, i.e., OLS, is rarely used in pharmacokinetic analysis, it offers the principle of least squares. The objective function of OLS can be regarded as a distance between observed and theoretical pharmacokinetic values on the Euclidean space RN, where N is the number of observations. Thus OLS produces its estimates by minimizing the Euclidean distance. On the other hand, MRE works by minimizing the relative entropy which expresses discrepancy between two probability densities. Because pharmacokinetic functions are not density function in general, we use a particular form of the relative entropy whose domain is extended to the space of all positive functions. MRE never assumes any distribution of errors involved in observations. Thus, it can be a possible solution to the choice of weights problem. Moreover, since the mathematical form of the relative entropy, i.e., an expectation of the log-ratio of two probability density functions, is different from that of a usual Euclidean distance, the behavior of MRE may be different from those of least squares methods. To clarify the behavior of MRE, we have compared the performance of MRE with those of ELS and OLS by carrying out an intensive simulation study, where four pharmaco-kinetic models (mono- or biexponential, Bateman, Michaelis-Menten) and several variance models for distribution of observation errors are employed. The relative precision of each method was investigated by examining the absolute deviation of each individual parameter estimate from the known value. OLS is the best method and MRE is not a good one when the actual observation error magnitude conforms to the assumption of OLS, that is, error variance is constant, but OLS always behaves poorly with the other variance models. On the other hand, MRE performs better than ELS and OLS when the variance of observation is proportional to its mean. In contrast, ELS is superior to MRE and OLS when the standard deviation of observation is proportional to its mean. In either case the difference between MRE and ELS is relatively small. Generally, the performance of MRE is comparable to that of ELS. Thus MRE provides as reliable a method as ELS for estimating pharmacokinetic parameters.

Martian magnetic moment evaluated by different methods on the basis of Phobos 2 measurements

Abstract Spectral analysis of the Phobos 2 magnetic field measurements by the method of global minimum and maximum entropy method was employed to evaluate the amplitude of the magnetic field in the tail connected with the corotating part of the intrinsic magnetic field. That made it possible to estimate the Martian moment M ≤ 1.25 10 12 Tm 3 which is uncertain by a factor 2 due to unknown angle between the spin axis and the magnetic moment. A second way of estimating the magnetic moment is from pressure balance near the magnetopause. Third - by calculation of the internal magnetic field in vicinity of the magnetopause. Comparison shows that the three calculated values coincide under some theoretical assumptions. This analysis can be applied to any planet.

Tail entropy approximations

A three parameter tail entropy approximation ( TEA) suitable for reliability and risk analysis is described and illustrated by examples. The TEA is determined directly from sample data. It combines the best features of tail approximation and entropy optimization for structural safety analysis. Estimation of the distributions of basic random variables from sample data is a critical step in the analysis of structural safety and other random phenomena. Standard estimation methods may be simple, but results are sensitive to the necessary but arbitrary assumptions about distribution type. The method of minimum relative entropy is more objective and less sensitive to arbitrary assumptions, but requires optimization procedures and some of the results are mathematically intractable. The TEA method is a tractable and accurate combination, useful when a tail approximation is appropriate.

Classification of faults and switching events by inductive reasoning and expert system methodology

Under certain conditions, one electrical parameter (independent variable) is not enough to detect high-impedance faults on certain surface conditions. These faults do not draw sufficient current for detection and may draw less current than similar faults on other soil surfaces. Moreover, because every electrical detection parameter displays characteristics of randomness, it is difficult to assign a probability that a given event is a high-impedance fault, rather than a switching event. It has been shown that detection by induction laws can improve the classification of faults and switching events. The second and third laws of induction are utilized with a minimum entropy method. Setting detection threshold values using induction methods is also proposed. The methods presented are taken from ongoing research in high-impedance fault detection. While the techniques have not been reduced to practice or field-tested, they hold promise for future improvements in the relaying of high-impedance faults.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Classification of Faults and Switching Events by Inductive Reasoning and Expert System Methodology

Under certain conditions, one electrical parameter (independent variable) is not enough to detect high-impedance faults on certain surface conditions. These faults do not draw sufficient current for detection and may draw less current than similar faults on other soil surfaces. Moreover, because every electrical detection parameter displays characteristics of randomness, it is difficult to assign a probability that a given event is a high-impedance fault, rather than a switching event. It has been shown that detection by induction laws can improve the classification of faults and switching events. The second and third laws of induction are utilized with a minimum entropy method. Setting detection threshold values using induction methods is also proposed. The methods presented are taken from ongoing research in high-impedance fault detection. While the techniques have not been reduced to practice or field-tested, they hold promise for future improvements in the relaying of high-impedance faults. >