Shannon's Measure Research Articles

Bounding Quantum Correlations: The Role of the Shannon Information in the Information Causality Principle.

The Information Causality principle was proposed to re-derive the Tsirelson bound, an upper limit on the strength of quantum correlations, and has been suggested as a candidate law of nature. The principle states that the Shannon information about Alice's distant database gained by Bob after receiving an m bit message cannot exceed m bits, even when Alice and Bob share non-local resources. As originally formulated, it can be shown that the principle is violated exactly when the strength of the shared correlations exceeds the Tsirelson bound. However, we demonstrate here that when an alternative measure of information, one of the Renyi measures, is chosen, the Information Causality principle no longer arrives at the correct value for the Tsirelson bound. We argue that neither the assumption of particular 'intuitive' properties of uncertainties measures, nor pragmatic choices about how to optimise costs associated with communication, are sufficient to motivate uniquely the choice of the Shannon measure from amongst the more general Renyi measures. We conclude that the dependence of the success of Information Causality on mere convention undermines its claimed significance as a foundational principle.

Mfind: a tool for DNA barcode analysis in angiosperms and its relationship with microsatellites using a sliding window algorithm.

Mfind is a tool to analyze the impact of microsatellite presence on DNA barcode specificity. We found a significant correlation between barcode entropy and microsatellite count in angiosperm. Genetic barcodes and microsatellites are some of the identification methods in taxonomy and biodiversity research. It is important to establish a relationship between microsatellite quantification and genetic information in barcodes. In order to clarify the association between the genetic information in barcodes (expressed as Shannon's Measure of Information, SMI) and microsatellites count, a total of 330,809 DNA barcodes from the BOLD database (Barcode of Life Data System) were analyzed. A parallel sliding-window algorithm was developed to compute the Shannon entropy of the barcodes, and this was compared with the quantification of microsatellites like (AT)n, (AC)n, and (AG)n. The microsatellite search method utilized an algorithm developed in the Java programming language, which systematically examined the genetic barcodes from an angiosperm database. For this purpose, a computational tool named Mfind was developed, and its search methodology is detailed. This comprehensive study revealed a broad overview of microsatellites within barcodes, unveiling an inverse correlation between the sumz of microsatellites count and barcodes information. The utilization of the Mfind tool demonstrated that the presence of microsatellites impacts the barcode information when considering entropy as a metric. This effect might be attributed to the concise length of DNA barcodes and the repetitive nature of microsatellites, resulting in a direct influence on the entropy of the barcodes.

Sampling-based epoch differentiation calibrated graph convolution network for point-of-interest recommendation

In location-based social networks, calibrating a point-of-interest (POI) recommendation system is as important as its accuracy for improving user satisfaction. POI recommendation calibration is primarily classified as categorical or geographical calibration. Categorical calibration ensures that the recommended items are distributed proportionally among the past interest categories of the target user. When a target user checks 80 Chinese, 10 Japanese, and 10 French restaurants, a recommendation list with a ratio of 8:1:1 for Chinese, Japanese, and French restaurants can be reasonably expected. In addition to categorical calibration, geographical calibration has been proposed to increase user interest in the recommended results. Users have a high probability of revisiting locations in their subareas. Therefore, the POIs recommended in multiple subareas of interest are more suitable than those from one small and frequently visited subarea. However, improving the calibration and accuracy are conflicting tasks. To achieve high calibration while maintaining accuracy, previous studies proposed reranking-based techniques to rerank the candidate list and return POIs with high calibration. However, optimizing the calibration by reranking is independent of the basic-candidate-item generation model, resulting in a suboptimal system. To tackle the problem, we propose a novel sampling-based differentiation technique to merge the task of improving calibration into the GCN model training process and directly generate the final recommendation list. The model is flexible and can be applied to different domains, where a domain can be a subarea or category. In a three-layer GCN, the layer one represents the historical check-ins of the user, whereas layer three includes the candidate POIs from which the target user aggregates information. We trained the model to make the distribution of the POI domains at layer three approximated the distribution at layer one. Experimental results on Philadelphia and Tucson datasets confirmed that the proposed method outperforms all state-of-the-art GCN+ geo-reranking and GCN+ MCF baselines, improving Recall@ 5 from 0.0394 to 0.0412 (4.57%) and Jensen–Shannon measure (JS)@ 5 from 0.5931 to 0.6734 (13.54%) on the Philadelphia dataset and improving Recall@ 5 from 0.0495 to 0.0517 (4.40%) and JS@ 5 from 0.5869 to 0.6598 (12.42%) on the Tucson dataset for categorical calibration. The model was also tested in the geographical domain and a similar trend was observed.

Understanding complexity of hydrogen-bonded liquids through Voronoi entropy

In this paper we study the short order of liquids using Voronoi entropy (SVor), computed using Voronoi Polyhedra (VP) tessellations. Voronoi tessellations partition a plane with n seeds into polyhedrons, in which each polyhedron has exactly one seed and every location within a given polyhedron is nearest to its generating seed, rather than to any other seed. The distribution of these VP is used to determine the Voronoi entropy (SVor), which shares similarities with the Shannon measure of information (H). Such studies, however, have been largely restricted to two-dimensional analysis. In this study SVor is used to quantify the orderliness of the three-dimensional structures of hydrogen-bonded systems (water, methanol, ethanol, and 1-propanol), as well as binary mixtures (methanol-ethanol, methanol-1-propanol, and ethanol-1-propanol) through molecular dynamics simulations. SVor is calculated for the VP considering all atom types and oxygen atoms to exclusively study the hydrogen bonds. The SVor for all atoms shows that the smaller molecules have a more ordered structure than the larger ones whereas the SVor calculated considering only oxygen atoms decreases as the alkyl chain length of the molecules increases indicating the smaller molecules have more ways to form hydrogen bonds than the larger ones. SVor calculated for binary mixtures for varying concentrations shows a similar trend.

Zeolite ATN: Topological characterization and predictive analysis on potential energies using entropy measures

Zeolites serve as multifunctional microporous materials and have potential applications as catalysts, ion exchangers, and adsorbents. Researchers are interested in the topological characterization of zeolite frameworks due to their complex 3D arrangement of atoms. ATN is one of the variants of zeolite whose building block is arranged in a non-isometric cyclic permutation. The long-range and short-range molecular interactions of the ATN framework are influenced by the geometric arrangement of molecules. In this paper, we have investigated Shannon's information entropy measures of the ATN framework using bond degree and neighborhood degree sum based indices as weights. Further, a predictive analysis with the aid of entropy measures to determine the long-range and short-range potential energies has been implemented. The model proposed in this study can be incorporated into the fields of molecular physics and theoretical chemistry to determine the energy levels of microporous materials. Therefore, the predictive model will make a substantial contribution to QSAR/QSPR modeling.

Cumulative α-Jensen–Shannon measure of divergence: Properties and applications

The problem of quantifying the distance between distributions arises in various fields, including cryptography, information theory, communication networks, machine learning, and data mining. In this article, the analogy with the cumulative Jensen–Shannon divergence, defined in Nguyen and Vreeken (2015), we propose a new divergence measure based on the cumulative distribution function and call it the cumulative α-Jensen–Shannon divergence, denoted by CJS ( α ) . Properties of CJS ( α ) are studied in detail, and also two upper bounds for CJS ( α ) are obtained. The simplified results under the proportional reversed hazard rate model are given. Various illustrative examples are analyzed.

Rational inattention with multiple attributes

This paper studies a new measure for the cost of learning that allows the different attributes of the options faced by an agent to differ in their associated learning costs. The new measure maintains the tractability of Shannon's classic measure but produces richer choice predictions and identifies a new form of informational bias significant for welfare and counterfactual analysis that is conducted with the multinomial logit model. Necessary and sufficient conditions are provided for optimal agent behavior under the new measure for the cost of learning.

Information, Entropy, Life, and the Universe.

In (2015), I wrote a book with the same title as this article. The book's subtitle is: "What we know and what we do not know." On the book's dedication page, I wrote [1]: "This book is dedicated to readers of popular science books who are baffled, perplexed, puzzled, astonished, confused, and discombobulated by reading about Information, Entropy, Life and the Universe." In the first part of this article, I will present the definitions of two central concepts: the "Shannon measure of information" (SMI), in Information Theory, and "Entropy", in Thermodynamics. Following these definitions, I will discuss the framework of their applicability. In the second part of the article, I will examine the question of whether living systems and the entire universe are, or are not within the framework of applicability of the concepts of SMI and Entropy. I will show that much of the confusion that exists in the literature arises because of people's ignorance about the framework of applicability of these concepts.

Information-entropy approach to the analysis of genetic diversity of populations (analytical review)

The application of information-entropy analysis on real data is shown: 84 bulls of Jersey (n = 10), Ayrshire (n = 10), red Danish (n = 10), red Swedish (n = 9) breeds and Holstein ecotypes of German (n = 13), Dutch (n = 17) and North American (n = 15) breeding genotyped by 11 microsatellite loci. The Shannon information index for the combined breed samples was 1.695, the weighted average for the samples was 1.325, the inter-sample component (SHAP) was 0.370 nits or 21.8 % (all calculations according to GenAlEx 6.502). The total absolute diversity of Shannon was 5.45, averaged by samples – 3.76 effective alleles per locus, between breeds – 1.45 effective samples. The relative estimate of the differentiation of breed samples (D'β) was 36.2 % and was close to the «traditional» estimates for Hedrick, Jost and Ciao. The influence of within samples heterozygosity on SHAP estimates has not been established, but a positive trend of D'β-estimates has been noted. The matrix of paired genetic distances by breeds (D'β) and its 2D projection had a high correspondence with those by GST(NEI)-, FST(W&C)-, G"ST(HED)-, F'ST(W&C)-, DJOST-, DCHAO-measures (RMantel ≥ 0,94). Fixation indices (GST(NEI) and FST(W&C)) should be used to study the evolutionary stories of subdivided populations. To analyze the current structure of the genetic diversity of (sub)populations with an emphasis on dominant allele measures based on heterozygosity (G'ST(HED), F'ST(W&C), DJOST and DCHAO), if necessary, take into account all alleles – the Shannon measure (D'β). In conclusion, a family of Hill power measures of order q = 0, 1, 2 is considered for constructing diversity profiles containing all information about the distribution of alleles in a population.

Shannon (Information) Measures of Symmetry for 1D and 2D Shapes and Patterns

In this paper, informational (Shannon) measures of symmetry are introduced and analyzed for patterns built of 1D and 2D shapes. The informational measure of symmetry Hsym(G) characterizes the averaged uncertainty in the presence of symmetry elements from group G in a given pattern, whereas the Shannon-like measure of symmetry Ωsym(G) quantifies the averaged uncertainty of the appearance of shapes possessing a total of n elements of symmetry belonging to group G in a given pattern. Hsym(G1)=Ωsym(G1)=0 for the patterns built of irregular, non-symmetric shapes, where G1 is the identity element of the symmetry group. Both informational measures of symmetry are intensive parameters of the pattern and do not depend on the number of shapes, their size, and the entire area of the pattern. They are also insensitive to the long-range order (translational symmetry) inherent for the pattern. Additionally, informational measures of symmetry of fractal patterns are addressed, the mixed patterns including curves and shapes are considered, the time evolution of Shannon measures of symmetry are examined, the close-packed and dispersed 2D patterns are analyzed, and an application of the suggested measures of symmetry for the analysis of the chemical reaction is demonstrated.

On the analysis of power law distribution in software component sizes

AbstractComponent‐based software development (CBSD) is an active area of research. Ascertaining the quality of components is important for overall software quality assurance in CBSD. One of the important metrics for measuring defects, analyzability, efforts, and cost in CBSD is component size. The paper presents an analytical model based on maximization of Tsallis entropy to obtain closed form expression for component size distribution (maximum Tsallis entropy component size distribution, MTECSD) in steady state. It is found that the component size distribution follows power law asymptotically. A procedure based on generalized Jensen–Shannon measure is developed to estimate model parameters. A detailed analysis of many popular probability distributions along with MTECSD is carried out on many diverse real data sets of component‐based softwares. The analysis reveals that lognormal and MTECSD distributions fit well to component sizes in many software conforming the presence of power law behavior. The software whose component size distributions are described by MTECSD are in equilibrium implying that new defects in these software systems occur occasionally. Power law behavior in component sizes also imply high variation leading to difficulty in software analyzability. The precise knowledge of component size distribution also provides an alternative method to compute efforts and cost estimates by modified COCOMO model.

The General Maximum Entropy Method by Shannon and Tsallis Measures for Estimating Parameters of a Kink Regression Model

In this paper, the parameters of the Kink regression model were estimated using the general maximum entropy method with the Shannon and Tsallis measures And its application for the analysis of economic exposure data and the Debt/GDP ratio in the Iraqi economy. The results of the analysis showed the preference of the Tsallis measure of the order (α = 3) compared with other estimation methods. The results show a decrease in the Debt/GDP ratio after seeing the cut, and this corresponds to the state of the economy in Iraq during the security conditions that Iraq went through for the period from 2014 to 2017.

Configurational Entropy: An Application Involving Census Tract Data for the City of Birmingham, Alabama

A variation of Shannon's relative entropy statistic is presented as a measure of configurational entropy for variables known to exhibit spatial autocorrelation using census tract data for the city of Birmingham, Alabama. Standardized and non-standardized configurational entropy indices (CEIs) are introduced to measure the amount of spatial order in a geographic distribution. As the degree of spatial autocorrelation increases and the amount of entropy or uncertainty decreases, the CEIs produce values that diverge from Shannon's statistic, which tends to overstate the degree of disorder or uncertainty in the presence of spatial autocorrelation. The CEIs incorporate a spatial covariance approach to estimating spatial order based on connectivity and the differencing of values for adjacent areal units. While Shannon's entropy statistic is insensitive to spatial arrangement, the CEIs provide a scale-targeted quantification of the amount of inherent spatial order in a distribution as defined by the connective structure of the areal units and the degree to which a variable is spatially autocorrelated. The amount of spatial order, as manifested within an autocorrelated pattern at a given geographic scale and for a given connectivity structure, is directly proportional to the difference between Shannon's measure and the CEIs.

Instance theory predicts information theory: Episodic uncertainty as a determinant of keystroke dynamics.

How does prior experience shape skilled performance in structured environments? We use skilled typing of natural text to evaluate correspondence between performance (keystroke timing) and structure in the environment (letter uncertainty). We had ∼350 typists copy-type English text. We reproduced Ostry's (1983) analysis of interkeystroke interval as a function of letter position and word length, that showed prominent first-letter and midword slowing effects. We propose a novel account that letter position and word length effects on keystroke dynamics reflect informational uncertainty about letters in those locations, rather than resource limited planning/buffering processes. We computed positional uncertainty for letters in all positions of words from length one to nine using Google's n-gram database. We show that variance in interkeystroke interval by letter position and word length tracks natural variation in letter uncertainty. Finally, we provide a model showing how a general learning and memory process could acquire sensitivity to patterns of letter uncertainty in natural English. In doing so, we draw an equivalence between Logan's (1988) instance theory of automatization and Shannon's measure of entropy (H) from information theory. Instance theory's predictions for automatization as a function of experience follow exactly the uncertainty in the choice set being automatized. As a result, instance theory stands as a general process model explaining how context-specific experiences in a structured environment tune skilled performance. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Information Across the Ecological Hierarchy

The ecosystem is a theatre upon which is presented, in various degrees and at differing scales, a drama of constraint and information vs. disorganization and entropy. Concerning biology, most think immediately of genomic information. It strongly constrains the form and behavior of individual species, but its influence upon community structure is indeterminate. At the community level, information acts as a formal cause behind regular patterns of development. Community structure is an amalgam of information and entropy, and the Gibbs–Boltzmann formula departs from the thermodynamic sense of entropy. It measures only the extreme that entropy might reach if the elements of the system were completely independent. A closer analogy to physical entropy in systems with interactions is the conditional entropy—the amount by which the Shannon measure is reduced after the information in the constraints among elements has been subtracted. Finally, at the whole ecosystem level, in communities that inhabit mostly fixed physical environments (e.g., landscapes or seabeds), the distributions of plants and animals appear to be independent both of causal mechanisms and trophic controls, and assume instead forms that maximize the overall entropy of dispersal.

Symmetry and Shannon Measure of Ordering: Paradoxes of Voronoi Tessellation.

The Voronoi entropy for random patterns and patterns demonstrating various elements of symmetry was calculated. The symmetric patterns were characterized by the values of the Voronoi entropy being very close to those inherent to random ones. This contradicts the idea that the Voronoi entropy quantifies the ordering of the seed points constituting the pattern. Extension of the Shannon-like formula embracing symmetric patterns is suggested. Analysis of Voronoi diagrams enables the elements of symmetry of the patterns to be revealed.

Is the Voronoi Entropy a True Entropy? Comments on "Entropy, Shannon's Measure of Information and Boltzmann's H-Theorem", Entropy 2017, 19, 48.

The goal of this comment note is to express our considerations about the recent paper by A. Ben Naim (Entropy 2017, 19, 48). We strongly support the distinguishing between the Shannon measure of information and the thermodynamic entropy, suggested in the paper. We demonstrate that the Voronoi entropy should also be clearly distinguished from the entropy of a two-dimensional gas. Actually, the Voronoi entropy being an intensive value is the averaged Shannon measure of ordering for a given pattern.

Storage protein variability in natural populations of maté (Ilex paraguariensis) in Brazil

ABSTRACT: Samples of 168 maté trees (Ilex paraguariensis) from four natural populations of different states of Brazil (Rio Grande do Sul, Santa Catarina, Paraná and Mato Grosso do Sul) were analyzed for their variability in storage proteins seed, which were used to estimate the degree of differentiation among and within species, because are relatively stable during evolution. Data on band presence/absence from 80 different polypeptides were used to perform similarity coefficient of Jaccard. A high level of genetic variability was reported within sampled populations (SJ intra=0.374). Lower levels of diversity were observed between populations (SJ inter=0.308). Total genetic diversity of maté was Hsp=0.264 estimated by Shannon measure. Partition of that value disclosed that 95 % of the total diversity is within-population, and only 5% to between-populations. The phenogram based on King’s distances indicates a certain degree of geographic differentiation. This represents the first study on storage protein variability in I. paraguariensis and guides the choice of the specimens to increment germoplasm banks and genetic improvement programs.

English

The roles of woody vegetations and Africa savanna in human development and survival as attracted interest in their conservation to prevent the depletion or loss of those resources. However, there is need for accurate data on species composition, distribution and conservation of woody species in many parts of Africa for adequate planning, monitoring, management and conservation efforts. This study assessed woody species composition, distribution and diversity in Kwara State University, Malete Campus. Simple random sampling technique was applied using plot method which was achieved using geographic information system (GIS) application to overlay the area boundary with grids of cell of 100 x 100 m plots. Data were collected from each plot, all woody species were identified, counted, and trees basal covers were measured. Species frequency, density, abundant, dominance, importance value index (IVI) and diversity were determined. A total of 46 trees and 10 shrubs species belonging to 20 families of tree and 8 families of shrubs were identified. Abundant tree species were Daniella oliveri and Azadirachata indica while the abundant shrubs species were Piliostigma thonningii and Acacia nilotica. Shannon diversity index and Shannon measure of evenness revealed that the diversity for trees species was higher (H’=2.4309 and J= 0.6349) than shrubs species (H’=1.1166 and J= 0.4849). There was spatial variation in diversity of trees and shrubs within the university which has more tree species than shrubs species. Hence, university management and community should pay attention to conservation planning and management activities with special consideration on their ecological implication. Key words: Diversity, tropical forest, savanna ecosystem, humid savanna, D. oliveri, A. indica, P. thonningii.

On selecting evidence to test hypotheses: A theory of selection tasks.

How individuals choose evidence to test hypotheses is a long-standing puzzle. According to an algorithmic theory that we present, it is based on dual processes: individuals' intuitions depending on mental models of the hypothesis yield selections of evidence matching instances of the hypothesis, but their deliberations yield selections of potential counterexamples to the hypothesis. The results of 228 experiments using Wason's selection task corroborated the theory's predictions. Participants made dependent choices of items of evidence: the selections in 99 experiments were significantly more redundant (using Shannon's measure) than those of 10,000 simulations of each experiment based on independent selections. Participants tended to select evidence corresponding to instances of hypotheses, or to its counterexamples, or to both. Given certain contents, instructions, or framings of the task, they were more likely to select potential counterexamples to the hypothesis. When participants received feedback about their selections in the "repeated" selection task, they switched from selections of instances of the hypothesis to selection of potential counterexamples. These results eliminated most of the 15 alternative theories of selecting evidence. In a meta-analysis, the model theory yielded a better fit of the results of 228 experiments than the one remaining theory based on reasoning rather than meaning. We discuss the implications of the model theory for hypothesis testing and for a well-known paradox of confirmation. (PsycINFO Database Record