Principle Of Minimum Cross-entropy Research Articles

Entropy-based Model for Gully Erosion - A combination of probabilistic and deterministic components.

Gullies are a major threat to ecosystems, potentially leading to land degradation, groundwater depletion, crop loss, debris flow, and desertification. Gullies are also characterized by having a fast development and turning into primary sediment sources. Despite their impact, we have but scarce understanding of how gully erosion evolves and how to model it. In this paper, we propose a new gully erosion model that is based on the classical premise of net shear stress, i.e., hydraulic shear stress minus critical (resistant) shear stress, to calculate detachment rates. In order to calculate hydraulic shear stress, we developed a new equation derived from the principle of minimum cross-entropy; it was validated with laboratory measures from the literature with a Nash-Sutcliffe Efficiency of 0.95. Soil samples were analysed in the laboratory to assess critical shear stress and other soil properties. The novel gully erosion model was implemented in three gully impacted locations with catchment areas ranging from 10-2 to 10+1 ha. To assess channel geometry and eroded volumes, we used Unmanned Aerial Vehicle and Structure-from-Motion technique. The model successfully estimated long-term erosion rates, its efficiency was 0.77, and it is recommended for catchments up to 8 ha. Therefore, the new model provides planners and stakeholders with a tool to assess gully erosion, sediment yield and geometry in most areas.

Coupling minimum cross-entropy model with experimental data to determine the drop size distribution for lanthanum extraction in ARDC column

ABSTRACT The influence of reactive extraction on the drop size distribution for lanthanum extraction was studied in the pilot plant asymmetric rotating disc column. The effect of agitation speed, the flow rate of aqueous and organic phases were investigated on the Sauter mean drop size and size distribution, extraction efficiency, and stripping percent. The prediction of drop size distribution was performed by using the minimum cross-entropy principle (MinCE). The higher values of extraction efficiency and stripping percent indicated that this column is the appropriate equipment for extraction. The outcome of this study provides valuable insights for the prediction of size distribution.

Predicting large-scale habitat suitability for cetaceans off Namibia using MinxEnt

MEPS Marine Ecology Progress Series Contact the journal Facebook Twitter RSS Mailing List Subscribe to our mailing list via Mailchimp HomeLatest VolumeAbout the JournalEditorsTheme Sections MEPS 619:149-167 (2019) - DOI: https://doi.org/10.3354/meps12934 Predicting large-scale habitat suitability for cetaceans off Namibia using MinxEnt P. De Rock1, S. H. Elwen1,2, J. P. Roux3,4, R. H. Leeney5, B. S. James1, V. Visser6,7, M. J. Martin2, T. Gridley1,6,* 1Sea Search Research and Conservation NPC, 4 Bath Rd, Muizenberg, Cape Town 7945, South Africa 2Mammal Research Institute, Department of Zoology and Entomology, University of Pretoria, South Africa 3Lüderitz Marine Research, Ministry of Fisheries and Marine Resources, Lüderitz, Namibia 4SEACODE, Box 583, Lüderitz, Namibia 5Natural History Museum, Cromwell Road, Kensington, London SW7 5BD, UK 6Statistics in Ecology, Environment and Conservation, Department of Statistical Sciences, University of Cape Town, Rondebosch, Cape Town 7701, South Africa 7African Climate and Development Initiative, University of Cape Town, Rondebosch 7701, South Africa *Corresponding author: tessgridley@yahoo.co.uk ABSTRACT: Knowledge of the occurrence and distribution of cetaceans is particularly important for conservation and management, but is still limited within Namibian waters. We collated 3211 cetacean records from the Namibian Exclusive Economic Zone (EEZ) for the period 2008 to 2016 and applied the principle of minimum cross entropy (MinxEnt) to predict habitat suitability. MinxEnt is a generalised form of maximum entropy modelling that allows incorporation of additional information such as sampling bias. The habitat suitability of 9 cetacean species or species groups (5 odontocete species, 2 mysticete species and 2 taxonomic groups: pilot whales Globicephalus spp. and balaenopterids Balaenopteridae spp.) were predicted per season, in relation to environmental variables likely to drive cetacean presence: sea surface temperature, chlorophyll a concentration, water depth or distance to shore, seabed slope and habitat complexity. The environmental variable which most frequently influenced habitat suitability was depth, which was the main environmental driver for bottlenose dolphin Tursiops truncatus, humpback Megaptera novaeangliae and southern right whales Eubalaena australis. Further, Heaviside’s dolphin Cephalorhynchus heavisidii habitat was best predicted by distance to shore in all seasons, while common dolphin Delphinus delphis and the balaenopterid group habitats were best predicted by habitat complexity, and sperm whale Physeter macrocephalus habitats by chlorophyll a concentration. We identify distinct spatial patterns in habitat suitability for different species and provide baseline maps which can be used by managers of wildlife resources. KEY WORDS: Cetacean · Dolphin · Habitat modelling · Habitat suitability · Marine protected area · Marine spatial planning · MaxEnt · Whale Full text in pdf format Supplementary material PreviousNextCite this article as: De Rock P, Elwen SH, Roux JP, Leeney RH and others (2019) Predicting large-scale habitat suitability for cetaceans off Namibia using MinxEnt. Mar Ecol Prog Ser 619:149-167. https://doi.org/10.3354/meps12934 Export citation RSS - Facebook - Tweet - linkedIn Cited by Published in MEPS Vol. 619. Online publication date: June 04, 2019 Print ISSN: 0171-8630; Online ISSN: 1616-1599 Copyright © 2019 Inter-Research.

Application of relative entropy theory to streamwise velocity profile in open-channel flow: effect of prior probability distributions

Applying the concept of relative or cross-entropy and the principle of minimum cross-entropy, this study derives the velocity distribution in a wide open channel. Previous studies have employed Shannon entropy and the principle of maximum entropy to derive distributions of various flow variables, including velocity. Relative entropy is a generalized form of entropy and can specialize into Shannon entropy if the prior probability distribution is taken to be a uniform distribution. The prior distribution is often formulated, based on intuition, experience, or thought experiment. When deriving the velocity distribution in wide open channels, this study assumes four prior probability distributions and analyzes the effect of these assumed priors. It is found that a normal-type and a gamma-type prior can significantly influence the velocity profile, especially near the channel bed, and their prediction accuracies are superior to the previously obtained velocity distribution based on Shannon entropy. Furthermore, closed-form explicit analytical solutions are obtained for the nonlinear differential equations that arise while incorporating these priors. Experimental and field data are used to verify the derived velocity distributions, and an error analysis is carried out to evaluate their performance.

A Kullback–Leibler View of Maximum Entropy and Maximum Log-Probability Methods

Entropy methods enable a convenient general approach to providing a probability distribution with partial information. The minimum cross-entropy principle selects the distribution that minimizes the Kullback–Leibler divergence subject to the given constraints. This general principle encompasses a wide variety of distributions, and generalizes other methods that have been proposed independently. There remains, however, some confusion about the breadth of entropy methods in the literature. In particular, the asymmetry of the Kullback–Leibler divergence provides two important special cases when the target distribution is uniform: the maximum entropy method and the maximum log-probability method. This paper compares the performance of both methods under a variety of conditions. We also examine a generalized maximum log-probability method as a further demonstration of the generality of the entropy approach.

Fully probabilistic design of hierarchical Bayesian models

The minimum cross-entropy principle is an established technique for design of an unknown distribution, processing linear functional constraints on the distribution. More generally, fully probabilistic design (FPD) chooses the distribution—within the knowledge-constrained set of possible distributions—for which the Kullback-Leibler divergence to the designer’s ideal distribution is minimized. These principles treat the unknown distribution deterministically. In this paper, fully probabilistic design is applied to hierarchical Bayesian models for the first time, yielding optimal design of a (possibly nonparametric) stochastic model for the unknown distribution. This equips minimum cross-entropy and FPD distributional estimates with measures of uncertainty. It enables robust choice of the optimal model, as well as randomization of this choice. The ability to process non-linear functional constraints in the constructed distribution significantly extends the applicability of these principles. Currently available FPD procedures for (a) merging of external knowledge, (b) approximate learning and stabilized forgetting, (c) decision strategy design, and (d) local adaptive control design, are unified for the first time via the hierarchical FPD framework of this paper.

Information-Theoretic-Entropy Based Weight Aggregation Method in Multiple-Attribute Group Decision-Making

Weight aggregation is the key process to solve a multiple-attribute group decision-making (MAGDM) problem. This paper is trying to propose a possible approach to objectivize subjective information and to aggregate information from attribute values themselves and decision-makers’ judgment. An MAGDM problem without information about decision-makers’ and attributes’ weight is considered. In order to define decision-makers’ subjective preference, their utility function is introduced. The attributes value matrix is converted into a subjective attributes value matrix based on their subjective judgment on attribute values. By utilizing the entropy weighting technique, decision-maker’s subjective weight on attributes and objective weight on attributes are determined individually based on the subjective attributes value matrix and attributes value matrix. Based on the principle of minimum cross-entropy, all decision-makers’ subjective weights are integrated into a single weight vector that is closest to all decision-makers’ judgment without any extra information added. Then, by applying the principle of minimum cross-entropy again, a weight aggregation method is proposed to combine the subjective and objective weight of attributes. Finally, an MAGDM example of project choosing is presented to illustrate the procedure of the proposed method.

Fuzzy cross-entropy

This paper deals with the divergence of fuzzy variables from a priori one. Within the framework of credibility theory, a fuzzy cross-entropy is defined to measure the divergence, and some mathematical properties are investigated. Furthermore, a minimum cross-entropy principle is proposed, which tells us that out of all membership functions satisfying given moment constraints, we should choose the one that is closest to the given a priori membership function.

Estimating Probability Distribution of Asset Value Based on Dual Effects of Deposit Insurance with the Minimum Cross-Entropy Principle

A model is proposed to estimate the probability distribution of asset value based on the benefit effects as well as the risk effects of deposit insurance with the minimum cross-entropy principle. Three scenarios are constructed to depict situations with different dual effects of deposit insurance. The corresponding assets distribution functions are obtained accordingly. The results show that it is positively correlated between the supervision level and the risk aversion effects of deposit insurance. The increase of the deposit insurance premium moves the bank’s assets distribution to the right side although not significantly. The asset probability distributions estimated in this paper can be taken as a reference for banks to choose the proper credit investment projects.

交互熵法在洪水频率分布参数估计中的应用研究

本文研究基于分位数对约束下的交互熵进行洪水频率分布参数估计方法。以加拿大Feather河和陕北地区张村驿站年最大洪峰流量序列为例，选取Gumbel和Gamma分布，基于最小交互熵原理，进行年最大洪峰流量序列分布参数估计，并根据估计参数推求洪峰流量频率曲线图。与矩法和极大似然法所求熵值比较，结果表明：交互熵法获得最小熵值，频率点距拟合亦取得满意效果。因此，在考虑分位数对约束的条件下，交互熵法能有效的估计分布参数，且较矩法和极大似然法优越。 This paper studies on the application of fractile constrained cross-entropy to the estimation of dis- tribution parameters in flood frequency analysis. Based on the principle of minimum cross-entropy, two annual maximum flood peak series respectively in Feather River in Canada and Zhangcunyi Station in north- ern Shaanxi province with Gumbel distribution and Gamma distribution were employed to the parameter estimation of the four distribution functions. Four frequency curves with the estimated parameters were also plotted. Then, comparing the calculated cross-entropy values with those that are derived by traditional methods- MOM and MLM, it turned out that: by using cross entropy method, we got the minimum cross entropy values. The plotted theoretical frequency curves fit well with the empirical frequency curves. So, we can conclude that the quantile constrained cross-entropy method has the considerable merit in the flood frequency parameter estimation and is superior to the traditional methods-MOM and MLM.

A New Entropy Optimization Model for Graduation of Data in Survival Analysis

Graduation of data is of great importance in survival analysis. Smoothness and goodness of fit are two fundamental requirements in graduation. Based on the instinctive defining expression for entropy in terms of a probability distribution, two optimization models based on the Maximum Entropy Principle (MaxEnt) and Minimum Cross Entropy Principle (MinCEnt) to estimate mortality probability distributions are presented. The results demonstrate that the two approaches achieve the two basic requirements of data graduating, smoothness and goodness of fit respectively. Then, in order to achieve a compromise between these requirements, a new entropy optimization model is proposed by defining a hybrid objective function combining both principles of MaxEnt and MinCEnt models linked by a given adjustment factor which reflects the preference of smoothness and goodness of fit in the data graduation. The proposed approach is feasible and more reasonable in data graduation when both smoothness and goodness of fit are concerned.

An efficient method for system reliability analysis of planar mechanisms

This article presents an efficient method for system reliability analysis of planar mechanisms with random dimensions and joint clearances. The reliability of the mechanism is defined as the probability that the output error remains within a specified limit in the entire target trajectory of the mechanism. Since the currently used methods for system reliability analysis are approximate in nature, this article presents a more efficient and innovative method based on the minimum cross-entropy principle for probabilistic analysis used in the information theory. The mechanism reliability problem is formulated as a series system reliability analysis that can be solved using the distribution of maximum output error. To derive this distribution, fractional moments of the output error are estimated from a small simulated sample of trajectories of mechanism motion which serve as constraints in the minimization of cross entropy. The proposed method is illustrated by analyzing the reliability slider–crank and a four-bar function generator. The comparison of the results with those obtained from the Monte Carlo simulations confirms the high accuracy of the proposed method.

An Approach to Predict Impact of Proposed Engineering Change Effect

Detailed evaluation of a proposed engineering change (EC) or its effects is a time-consuming process requiring considerable user experience and expertise. Therefore, enterprises plan detailed evaluation of only those EC effects that might have a significant impact. Since similar ECs are likely to have similar effects and impacts, past EC knowledge can be utilized for determining whether the proposed EC effect has significant impact. This paper presents an approach for predicting the impact of proposed EC effect based on past ECs that are similar to it. Our approach accounts for the differences in context of impact between attribute values in two changes. The Bayes’ rule is utilized to determine differences in impact value based on the differences in attribute values. The probability values required in Bayes’ rule are determined based on the principle of minimum cross entropy. An example EC knowledge base is created and utilized to evaluate our approach against two state-of-the-art approaches, namely k-nearest neighbor (NN) and regularized local similarity discriminant analysis (SDA). The success rate in predicting impact is used as an evaluation metric. The results show that there is a statistically significant improvement in success rate obtained using our approach as compared to those obtained using the two state-of-the-art approaches. The results also show that for a very large number of proposed ECs, i.e., N > 100, the success rate in predicting impact using our approach shall be greater than that obtained using the two state-of-the-art approaches.

Cross-entropy measure of uncertain variables

ross-entropy is a measure of the difference between two distribution functions. In order to deal with the divergence of uncertain variables via uncertainty distributions, this paper aims at introducing the concept of cross-entropy for uncertain variables based on uncertain theory, as well as investigating some mathematical properties of this concept. Several practical examples are also provided to calculate uncertain cross-entropy. Furthermore, the minimum cross-entropy principle is proposed in this paper. Finally, a study of generalized cross-entropy for uncertain variables is carried out.

Multivariate skew- t approach to the design of accumulation risk scenarios for the flooding hazard

The multivariate version of the skew- t distribution provides a powerful analytical description of the joint behavior of multivariate processes. It enjoys valuable properties: from the aptitude to model skewed as well as leptokurtic datasets to the availability of moments and likelihood analytical expressions. Moreover, it offers a wide range of extremal dependence strength, allowing for upper and lower tail dependence. The idea underneath this work is to employ the multivariate skew- t distribution to provide an estimation of the joint probability of flood events in a multi-site multi-basin approach. This constitutes the basis for the design and evaluation of flood hazard scenarios for large areas in terms of their intensity, extension and frequency, i.e. those information required by civil protection agencies to put in action mitigation strategies and by insurance companies to price the flooding risk and to evaluate portfolios. Performances of the skew- t distribution and the corresponding t copula function, introduced to represent the state of the art for multivariate simulations, are discussed with reference to the Tanaro Basin, North-western Italy. To enhance the characteristics of the correlation structure, three nested and non-nested gauging stations are selected with contributing areas from 1500 to 8000 km 2. A dataset of 76 trivariate flood events is extracted from a mean daily discharges database available for the time period from January 1995 to December 2003. Applications include the generation of multivariate skew- t and t copula samples and models' comparison through the principle of minimum cross-entropy, here revised for the application to multivariate samples. Copula and skew- t based scenario return period estimations are provided for the November 1994 flood event, i.e. the worst on record in the 1801–2001 period. Results are encouraging: the skew- t distribution seems able to describe the joint behavior, being close to the observations. Marginal distributions derived from the skew- t multivariate fit are comparable to the observed ones, and the model is also able to describe the tail behavior.

Cross entropy quantile function estimation from censored samples using partial probability weighted moments

Extreme quantile estimation using the minimum entropy principle from complete or non-censored samples has been reported in the literature. However, censored samples are often encountered in many engineering analysis such as annual extremes of river discharge and stream flow analysis. In this paper, the principle of minimum cross entropy (CrossEnt) is extended to partial minimum CrossEnt principle, which is defined on a finite interval. By interpreting the partial probability weighted moments (PPWMs) as partial moments of quantile function, the paper presents a new distribution free method for estimating the quantile function of a non-negative random variable using the principle of partial minimum CrossEnt subject to constraints on PPWMs estimated from censored data. Numerical examples are performed to assess the accuracy of extreme quantile estimates computed from censored samples.

Cross Entropy Approximation of Structured Gaussian Covariance Matrices

We apply two variations of the principle of minimum cross entropy (the Kullback information measure) to fit parameterized probability density models to observed data densities. For an array beamforming problem with P incident narrowband point sources, sensors, and colored noise, both approaches yield eigenvector fitting methods similar to that of the MUSIC algorithm and of the oblique transformation in factor analysis. Furthermore, the corresponding cross entropies (CE) are related to the MDL model order selection criterion .

Extreme quantile estimation from censored sample using partial cross-entropy and fractional partial probability weighted moments

Quantile function (QF) estimation by using minimum cross-entropy principle from complete or non-censored samples was reported before [Pandey MD. Extreme quantile estimation using order statistics with minimum cross-entropy principle. Probabilist Eng Mech 2001;16(1):31–42]. However, censored samples are often encountered in engineering reliability and hydrology distribution analysis. This paper presents a new distribution free method for estimating the quantile function of a non-negative random variable using the principle of partial minimum cross-entropy subject to constraints specified in terms of fractional partial probability weighted moments (FPPWMs) estimated from censored observed data. The proposed method exhibits considerable flexibility and covers two special cases. The numerical results show that substantial improvement in efficiency and accuracy of quantile estimation by use of the new method over other approaches.

Estimation of minimum cross-entropy quantile function using fractional probability weighted moments

The principle of minimum cross-entropy provides a systematic approach to derive the posterior distribution of a random variable given a prior and additional information in terms of its product moments. This approach can be extended to derive directly the quantile function by using probability weighted moments (PWMs) as constraints in the cross-entropy minimization approach, as shown in a previous study [Pandey MD. Extreme quantile estimation using order statistics with minimum cross-entropy principle. Probabilistic Engineering Mechanics 2001;16(1):31–42]. The objective of the present paper is to extend and improve the previous method by incorporating the use of the fractional probability weighted moments (FPWMs) in the place of conventional integer-order PWMs. A new and general estimation method is proposed in which the Monte Carlo simulations and optimization algorithms are combined to estimate FPWMs that would subsequently lead to the best-fit quantile function. The numerical examples presented in the paper show a substantial improvement in accuracy by the use of the proposed method over the conventional approach.

Analysis of wind speed distributions: Wind distribution function derived from minimum cross entropy principles as better alternative to Weibull function

In this study, the minimum cross entropy (MinxEnt) principle is applied for the first time to the wind energy field. This principle allows the inclusion of previous information of a wind speed distribution and covers the maximum entropy (MaxEnt) principle, which is also discussed by Li and Li and Ramirez as special cases in their wind power study. The MinxEnt probability density function (pdf) derived from the MinxEnt principle are used to determine the diurnal, monthly, seasonal and annual wind speed distributions. A comparison between MinxEnt pdfs defined on the basis of the MinxEnt principle and the Weibull pdf on wind speed data, which are taken from different sources and measured in various regions, is conducted. The wind power densities of the considered regions obtained from Weibull and MinxEnt pdfs are also compared. The results indicate that the pdfs derived from the MinxEnt principle fit better to a variety of measured wind speed data than the conventionally applied empirical Weibull pdf. Therefore, it is shown that the MinxEnt principle can be used as an alternative method to estimate both wind distribution and wind power accurately.