Shannon's Measure Research Articles

Efficient Data Assumption and Optimization of Analog Response Communication Systems

<span lang="EN-US">This paper is focused on the examination of connection between the premier remote reason exactitude of transmission information and qualities for driving edge correspondence structures and straightforward data cesium (AFCS) identified with transmission of signs from fundamental sources. It's displayed that the mean sq. oversight of transmission picks information properties of AFCS. Varieties between the cutoff purposes of AFCS thought of as summed up correspondence channel and their forward channel square measure investigated. The new impacts appearing in incredible AFCS and in DCS working near Shannon's cutoff square measure thought of also. Amid this works advanced and essential correspondence frameworks were thought of with none inclinations and no confirmation that the use of simple cesium is frequently less able than modernized correspondence structures had been given start inside the no in this way far off past. The secured comes about unambiguously showed the capacity of simple data cesium to transmit the signs while not committal to composing with to a little degree rate up to the uttermost ranges of the forward control.</span>

Urban growth prediction using neural network coupled agents-based Cellular Automata model for Sriperumbudur Taluk, Tamil Nadu, India

Unplanned urbanization would pose serious threats to both environment and mankind. Hence, urban growth model (UGM) becomes mandatory to predict future growth of a city. In the current study, urban growth of Sriperumbudur Taluk, Tamil Nadu, India was predicted using three types of Cellular Automata (CA) model namely Traditional CA (TCA) model, Agents based Cellular Automata (ACA) Model and Neural Network coupled Agents- based Cellular Automata (NNACA) model. The urban maps of the study region for the years 2009, 2013 and 2016 along with the influencing agents of urbanization namely transportation, industries, elevation and also hotspot locations based on the Government policy were used in the modeling. Analytical Hierarchical Process (AHP) technique was adopted to estimate the weights of the agents for suitability map preparation in ACA model. On validating 2016 predicted outputs, NNACA model proved to be the better urban model (kappa coefficient - 0.72) when compared to TCA and ACA models (kappa coefficient - 0.6 each). Shannon's entropy measure revealed that the urbanization is concentrated in the north-east direction and it is predicted to have an urban sprawl area of 157 km2 in 2020 using NNACA model while the observed urbanization is 113 km2 of the area in 2016.

Some properties of Lin–Wong divergence on the past lifetime data

ABSTRACTMeasures of statistical divergence are used to assess mutual similarities between distributions of multiple variables through a variety of methodologies including Shannon entropy and Csiszar divergence. Modified measures of statistical divergence are introduced throughout the present article. Those modified measures are related to the Lin–Wong (LW) divergence applied on the past lifetime data. Accordingly, the relationship between Fisher information and the LW divergence measure was explored when applied on the past lifetime data. Throughout this study, a number of relations are proposed between various assessment methods which implement the Jensen–Shannon, Jeffreys, and Hellinger divergence measures. Also, relations between the LW measure and the Kullback–Leibler (KL) measures for past lifetime data were examined. Furthermore, the present study discusses the relationship between the proposed ordering scheme and the distance interval between LW and KL measures under certain conditions.

Proposal for Measuring Quality of Decision Trees Partition

To compute a partition quality for a decision tree, we propose a new measure called NIM “New Information Measure”. The measure is simpler, provides similar performance, and sometimes outperforms the existing measures used with tree-based methods. The experimental results using the MONITDIAB application (Taleb & Atmani, 2013) and datasets from the UCI repository (Asuncion & Newman, 2007) confirm the classification capabilities of our proposal in comparison to the Shannon measure used with ID3 and C4.5 decision tree methods.

Reply to “Comments on ‘Statistical complexity and Fisher–Shannon information measure of [formula omitted]’ [Phys. Lett. A 372 (13) (2008) 2271–2273

Clarifications about the source of error in [Phys. Lett. A 371 (2008) 2271] of momentum-space information measures are given along with the revised estimates which satisfy all the points commented upon by Astashkevich. We note that the position-space quantities calculated previously remain unchanged. Our main conclusion, that the inter-nuclear distance, R, dependence of statistical complexity and Fisher–Shannon measure derived from the simple Coulson wave function contain significant information on molecular bonding in H2+, remains qualitatively valid in the light of calculations using more accurate wave functions as reported by Astashkevich.

The biparametric Fisher–Rényi complexity measure and its application to the multidimensional blackbody radiation

We introduce a biparametric Fisher–Rényi complexity measure for general probability distributions and we discuss its properties. This notion, which is composed of two entropy-like components (the Rényi entropy and the biparametric Fisher information), generalizes the basic Fisher–Shannon measure and the previous complexity quantifiers of Fisher–Rényi type. Then, we illustrate the usefulness of this notion by carrying out a information-theoretical analysis of the spectral energy density of a d-dimensional blackbody at temperature T. It is shown that the biparametric Fisher–Rényi measure of this quantum system has a universal character in the sense that it does not depend on temperature nor on any physical constant (e.g. Planck constant, speed of light, Boltzmann constant), but only on the space dimensionality d. Moreover, it decreases when d is increasing, but exhibits a non trivial behavior for a fixed d and a varying parameter, which somehow brings up a non standard structure of the blackbody d-dimensional density distribution.

Discussing Landscape Compositional Scenarios Generated with Maximization of Non-Expected Utility Decision Models Based on Weighted Entropies

The search for hypothetical optimal solutions of landscape composition is a major issue in landscape planning and it can be outlined in a two-dimensional decision space involving economic value and landscape diversity, the latter being considered as a potential safeguard to the provision of services and externalities not accounted in the economic value. In this paper, we use decision models with different utility valuations combined with weighted entropies respectively incorporating rarity factors associated to Gini-Simpson and Shannon measures. A small example of this framework is provided and discussed for landscape compositional scenarios in the region of Nisa, Portugal. The optimal solutions relative to the different cases considered are assessed in the two-dimensional decision space using a benchmark indicator. The results indicate that the likely best combination is achieved by the solution using Shannon weighted entropy and a square root utility function, corresponding to a risk-averse behavior associated to the precautionary principle linked to safeguarding landscape diversity, anchoring for ecosystem services provision and other externalities. Further developments are suggested, mainly those relative to the hypothesis that the decision models here outlined could be used to revisit the stability-complexity debate in the field of ecological studies.

Entropy, Shannon’s Measure of Information and Boltzmann’s H-Theorem

We start with a clear distinction between Shannon’s Measure of Information (SMI) and the Thermodynamic Entropy. The first is defined on any probability distribution; and therefore it is a very general concept. On the other hand Entropy is defined on a very special set of distributions. Next we show that the Shannon Measure of Information (SMI) provides a solid and quantitative basis for the interpretation of the thermodynamic entropy. The entropy measures the uncertainty in the distribution of the locations and momenta of all the particles; as well as two corrections due to the uncertainty principle and the indistinguishability of the particles. Finally we show that the H-function as defined by Boltzmann is an SMI but not entropy. Therefore; much of what has been written on the H-theorem is irrelevant to entropy and the Second Law of Thermodynamics.

An Application of Generalized Entropy Optimization Methods in Survival Data Analysis

In this paper, survival data analysis is realized by applying Generalized Entropy Optimization Methods (GEOM). It is known that all statistical distributions can be obtained as distribution by choosing corresponding moment functions. However, Generalized Entropy Optimization Distributions (GEOD) in the form of distributions which are obtained on basis of Shannon measure and supplementary optimization with respect to characterizing moment functions, more exactly represent the given statistical data. For this reason, survival data analysis by GEOD acquires a new significance. In this research, the data of the life table for engine failure data (1980) is examined. The performances of GEOD are established by Chi-Square criteria, Root Mean Square Error (RMSE) criteria and Shannon entropy measure, Kullback-Leibler measure. Comparison of GEOD with each other in the different senses shows that along of these distributions (MinMaxEnt)4 is better in the senses of Shannon measure and of Kullback-Leibler measure. It is showed that, (MinMaxEnt)3 ((MaxMaxEnt)4) is more suitable for statistical data among (MinMaxEnt)m,m=1,2,3,4(MaxMaxEnt)m,m=1,2,3,4. Moreover, (MinMaxEnt)3 is better for statistical data than (MaxMaxEnt)4 in the sense of RMSE criteria. According to obtained distribution (MinMaxEnt)3 (MaxMaxEnt)4 estimator of Probability Density Function f^ (t), Cumulative Distribution Functio F^ (t) , Survival Function Ŝ(t) and Hazard Rate ĥ(t) are evaluated and graphically illustrated. The results are acquired by using statistical software MATLAB.

Patterns of Gene Flow between Crop and Wild Carrot, Daucus carota (Apiaceae) in the United States.

Studies of gene flow between crops and their wild relatives have implications for both management practices for cultivation and understanding the risk of transgene escape. These types of studies may also yield insight into population dynamics and the evolutionary consequences of gene flow for wild relatives of crop species. Moreover, the comparison of genetic markers with different modes of inheritance, or transmission, such as those of the nuclear and chloroplast genomes, can inform the relative risk of transgene escape via pollen versus seed. Here we investigate patterns of gene flow between crop and wild carrot, Daucus carota (Apiaceae) in two regions of the United States. We employed 15 nuclear simple sequence repeat (SSR) markers and one polymorphic chloroplast marker. Further, we utilized both conventional population genetic metrics along with Shannon diversity indices as the latter have been proposed to be more sensitive to allele frequency changes and differentiation. We found that populations in both regions that were proximal to crop fields showed lower levels of differentiation to the crops than populations that were located farther away. We also found that Shannon measures were more sensitive to differences in both genetic diversity and differentiation in our study. Finally, we found indirect evidence of paternal transmission of chloroplast DNA and accompanying lower than expected levels of chloroplast genetic structure amongst populations as might be expected if chloroplast DNA genes flow through both seed and pollen. Our findings of substantial gene flow for both nuclear and chloroplast markers demonstrate the efficiency of both pollen and seed to transfer genetic information amongst populations of carrot.

Genes are information, so information theory is coming to the aid of evolutionary biology.

Speciation is central to evolutionary biology, and to elucidate it, we need to catch the early genetic changes that set nascent taxa on their path to species status (Via 2009). That challenge is difficult, of course, for two chief reasons: (i) serendipity is required to catch speciation in the act; and (ii) after a short time span with lingering gene flow, differentiation may be low and/or embodied only in rare alleles that are difficult to sample. In this issue of Molecular Ecology Resources, Smouse et al. (2015) have noted that optimal assessment of differentiation within and between nascent species should be robust to these challenges, and they identified a measure based on Shannon's information theory that has many advantages for this and numerous other tasks. The Shannon measure exhibits complete additivity of information at different levels of subdivision. Of all the family of diversity measures ('0' or allele counts, '1' or Shannon, '2' or heterozygosity, F(ST) and related metrics) Shannon's measure comes closest to weighting alleles by their frequencies. For the Shannon measure, rare alleles that represent early signals of nascent speciation are neither down-weighted to the point of irrelevance, as for level 2 measures, nor up-weighted to overpowering importance, as for level 0 measures (Chao et al. 2010, )2015. Shannon measures have a long history in population genetics, dating back to Shannon's PhD thesis in 1940 (Crow 2001), but have received only sporadic attention, until a resurgence of interest in the last ten years, as reviewed briefly by Smouse et al. (2015).

Discover probability: how to use it, how to avoid misusing it, and how it affects every aspect of your life

What is the Probability? How Do We Calculate Probabilities? The Axiomatic Approach to Probability Independence and Dependence Between Events Bayes' Theorem and Its Applications Average, Variance and Random Variable Probability Distributions Shannon's Measure of Information

On Applications of a Generalized Hyperbolic Measure of Entropy

After generalization of Shannon's entropy measure by Renyi in 1961, many generalized versions of Shannon measure were proposed by different authors. Shannon measure can be obtained from these generalized measures asymptotically. A natural question arises in the parametric generalization of Shannon's entropy measure. What is the role of the parameter(s) from application point of view? In the present communication, super additivity and fast scalability of generalized hyperbolic measure (Bhatia and Singh, 2013) of probabilistic entropy as compared to some classical measures of entropy has been shown. Application of a generalized hyperbolic measure of probabilistic entropy in certain situations has been discussed. Also, application of generalized hyperbolic measure of fuzzy entropy in multi attribute decision making have been presented where the parameter affects the preference order. i i i i i p p E Inf p P H       

Phase/current information descriptors and equilibrium states in molecules

Quantum‐generalized entropic descriptors of the complex electronic states and their information distances are reexamined and applied to the phase‐equilibria in molecules. The relation between densities of the ordinary Fisher and Shannon measures of information content is used in determining their supplements due to phases/currents. These nonclassical terms complement the familiar classical (probability) functionals of information theory in the resultant information descriptors. The nonclassical Shannon entropy measures the average magnitude of the system phase distribution, while the current term in the related Fisher measure accounts for the gradient content of the state phase. The density constrained (vertical) and unconstrained (horizontal) equilibria in molecules are distinguished. The consistency requirement that the extreme entropic principles in terms of both these resultant measures have common solutions calls for the modified, negative sign of the nonclassical Fisher indeterminicity term. The equilibrium criteria are shown to give rise to the unitary phase‐transformation of molecular states in a “thermodynamic” representation of quantum‐mechanical description. Possible applications of this generalized description are discussed and thermodynamical analogies are commented upon. A separation of the density (modulus) and current (phase) factors of general many‐electron states is effected using the Harriman–Zumbach–Maschke construction of antisymmetric states yielding the specified electron density. A phenomenological description of molecular subsystems is outlined, which accounts for both the density and phase degrees‐of‐freedom of electronic states, and the current promotion of molecular fragments is explored. © 2014 Wiley Periodicals, Inc.

Quantitative Robust Declassification

The previous declassification policies focus on qualitative analysis of sec urity properties along different dimensions, lacking quantitative analysis of them. As a step in this direction, we relax restrictiven ess of robustness of declassification from the quantitative aspect, and propose a definition of robustness rate of declassification, based on Shannon's measure method of information lattice. We show our definition is equivalent to robust declassification when the va lue of robustness rate is equal to 1. Moreover, we make a theoretical and experimental analysis of robust rate about the launder ing attack on average salary, respectively. The experimental results are consistent with the theoretical results completely.

Simulating Potential Population Growth of Wild Pig, Sus scrofa, in Texas

Understanding the interrelationship of environmental and biological factors that influence population growth rates of invasive Sus scrofa (Wild Pig) is a requisite for population management of the species. Such information can be used to evaluate various types of population control to ensure that the most cost-effective damage-abatement methods are used. We developed a sex- and age-structured model to simulate general population dynamics of Wild Pigs in Texas. Our objectives were to estimate potential statewide Wild Pig population-growth rates for Texas, identify model parameters that most influenced population trajectories, and compare resulting model predictions with ancillary population-trend data. Our Wild Pig simulation model estimated a mean annual growth rate of 0.32 (SE = 0.01), and stochastic model projections of Wild Pig population sizes ranged from 3.6 million to 16.9 million after 5 years. To evaluate parameter sensitivity, we recast our simulation results into a Bayesian belief network, and evaluated input-parameter influence based on variance reduction using Shannon's measure of mutual information. Our results indicated that the most influential model parameters within our simulation were number of litters per female and number of piglets recruited into the population, while adult and juvenile survival had little influence on Wild Pig population size within our simulations. Overall, our results suggest that natural resource managers should focus efforts towards reducing Wild Pig reproductive success, as opposed to attempting to increase adult mortality, when conducting Wild Pig population-control campaigns.

Entropy and the second law: interpretation and misss-interpretationsss

Introduction: From Heat Engines to Disorder, Information Spreading, Freedom, and More - Forget about Entropy for a While, Let us Go and Play iGames The Astounding Emergence of the Entropy of a Classical Ideal Gas out of Shannon's Measure of Information Examples and Their Interpretations. Challenges for any Descriptor of Entropy Finally, Let Us Discuss the Most Mysterious Second Law of Thermodynamics.

A Comprehensive Methodology for Determining the Most Informative Mammographic Features

This study aims to determine the most informative mammographic features for breast cancer diagnosis using mutual information (MI) analysis. Our Health Insurance Portability and Accountability Act-approved database consists of 44,397 consecutive structured mammography reports for 20,375 patients collected from 2005 to 2008. The reports include demographic risk factors (age, family and personal history of breast cancer, and use of hormone therapy) and mammographic features from the Breast Imaging Reporting and Data System lexicon. We calculated MI using Shannon's entropy measure for each feature with respect to the outcome (benign/malignant using a cancer registry match as reference standard). In order to evaluate the validity of the MI rankings of features, we trained and tested naïve Bayes classifiers on the feature with tenfold cross-validation, and measured the predictive ability using area under the ROC curve (AUC). We used a bootstrapping approach to assess the distributional properties of our estimates, and the DeLong method to compare AUC. Based on MI, we found that mass margins and mass shape were the most informative features for breast cancer diagnosis. Calcification morphology, mass density, and calcification distribution provided predictive information for distinguishing benign and malignant breast findings. Breast composition, associated findings, and special cases provided little information in this task. We also found that the rankings of mammographic features with MI and AUC were generally consistent. MI analysis provides a framework to determine the value of different mammographic features in the pursuit of optimal (i.e., accurate and efficient) breast cancer diagnosis.

Statistical measures of complexity for quantum systems with continuous variables

The Fisher–Shannon statistical measure of complexity is analyzed for a continuous manifold of quantum observables. It is shown that evaluating this measure only in the configuration or in the momentum spaces does not provide an adequate characterization of the complexity of some quantum systems. In order to obtain a more complete description of complexity two new measures, respectively based on the minimization and the integration of the usual Fisher–Shannon measure over all the parameter space, are proposed and compared. Finally, these measures are applied to the concrete case of a free particle in a box.

Early Fusion of Low Level Features for Emotion Mining

We study the discrimination of emotions annotated in free texts at the sentence level: a sentence can either be associated with no emotion (neutral) or multiple labels of emotion. The proposed system relies on three characteristics. We implement an early fusion of grams of increasing orders transposing an approach successfully employed in the related task of opinion mining. We apply a filtering process that consists in extracting frequent n-grams and making use of the Shannon’s entropy measure to respectively maintain dictionaries at balanced sizes and keep emotion specific features. Finally the overall system is implemented as a 2-step decision process: a first classifier discriminates between neutral and emotion bearing sentences, then one classifier per emotion is applied on emotion bearing sentences. The final decision is given by the classifier holding the maximum confidence. Results obtained on the testing set are promising.