Shannon Information Measures Research Articles

Genetic diversity and differentiation among populations of Bretschneidera sinensis (Bretschneideraceae), a narrowly distributed and endemic species in China, detected by inter-simple sequence repeat (ISSR)

Bretschneidera sinensis Hemsl., of the monogeneric family Bretschneideraceae, is an endangered species endemic in East and South China. Most of the populations are seriously threatened. Some of them are even at the brink of extinction. In this study, Inter-simple sequence repeat (ISSR) markers were used to investigate the genetic diversity within and among seven populations of B. sinensis. The results showed that Genetic diversity at species level was high (PPB: 76.7%; HE: 0.3131; H: 0.4524). However, relatively low genetic diversity existed within populations. Population in Guangdong (GD) exhibits the greatest level of variability (PPB: 65.1%; HE: 0.2573, H: 0.3766), whereas the population in Fengyangshan (FYS) finds its own variability at the lowest level (PPB: 51.5%; HE: 0.1959, H: 0.2886). Based on the genetic analyses of the ISSRs, a relatively high level of genetic differentiation among populations was revealed by Nei's gene diversity statistics (22.3%), Shannon's information measure (21.5%) and analysis of molecular variance (AMOVA) (18.4%). The main factors responsible for the relatively high level of differentiation among populations are probably related to the predominant outcrossing reproductive system and recent habitat fragmentation, and a significant correlation was found between genetic distance and geographic distance (r = 0.9087, P < 0.05). The high diversity found in this study holds promise for conservation and restoration efforts to save the endangered species from extinction.

Genetic diversity of Broussonetia papyrifera populations in southwest China.

Broussonetia papyrifera is an important native tree species with high economic value in southwest China. Its resources are drastically reduced because of over-harvesting and habitat fragmentation. In this study, 17 natural populations of B. papyrifera were analyzed using inter-simple sequence repeat (ISSR) markers to assess the genetic diversity and population structure. In total, 100 bands were obtained from 16 ISSR primers. The B. papyrifera populations showed relatively high genetic diversity at the species level [percentage of polymorphic bands (PPB): 96%; Nei's genetic diversity (HE): 0.3074; Shannon's information index (I): 0.4617], while the genetic diversity at the population level was relatively low (PPB: 53.2%; HE: 0.1826; I: 0.2735). Relatively high level of genetic differentiation among populations (41%) was disclosed by analysis of molecular variance, which agrees with the Nei's genetic diversity statistics (40.59%) and Shannon's information measure (40.76%). Gene flow among populations (NM) was only 0.7318. A significant correlation was observed between genetic and geographic distance among the studied populations (r=0.2948). We conjectured that the genetic diversity of B. papyrifera resulted from human disturbance, habitat fragmentation, small effective population size, and geographic barrier. Given the high genetic differentiation among populations, some utilization and conservation strategies were proposed. This study provides a reference for the sustainable use of the species in southwest China.

Axioms for (α, β, γ)-entropy of a generalized probability scheme

Abstract In this communication, we characterize a measure of information of type (α, β, γ) by taking certain axioms parallel to those considered earlier by Harvda and Charvat along with the recursive relation (1.7). Some properties of this measure are also studied. This measure includes Shannon information measure as a special case

A Bayesian trans-dimensional approach for the fusion of multiple geophysical datasets

We propose a Bayesian fusion approach to integrate multiple geophysical datasets with different coverage and sensitivity. The fusion strategy is based on the capability of various geophysical methods to provide enough resolution to identify either subsurface material parameters or subsurface structure, or both. We focus on electrical resistivity as the target material parameter and electrical resistivity tomography (ERT), electromagnetic induction (EMI), and ground penetrating radar (GPR) as the set of geophysical methods. However, extending the approach to different sets of geophysical parameters and methods is straightforward. Different geophysical datasets are entered into a trans-dimensional Markov chain Monte Carlo (McMC) search-based joint inversion algorithm. The trans-dimensional property of the McMC algorithm allows dynamic parameterisation of the model space, which in turn helps to avoid bias of the post-inversion results towards a particular model. Given that we are attempting to develop an approach that has practical potential, we discretize the subsurface into an array of one-dimensional earth-models. Accordingly, the ERT data that are collected by using two-dimensional acquisition geometry are re-casted to a set of equivalent vertical electric soundings. Different data are inverted either individually or jointly to estimate one-dimensional subsurface models at discrete locations. We use Shannon's information measure to quantify the information obtained from the inversion of different combinations of geophysical datasets. Information from multiple methods is brought together via introducing joint likelihood function and/or constraining the prior information. A Bayesian maximum entropy approach is used for spatial fusion of spatially dispersed estimated one-dimensional models and mapping of the target parameter. We illustrate the approach with a synthetic dataset and then apply it to a field dataset We show that the proposed fusion strategy is successful not only in enhancing the subsurface information but also as a survey design tool to identify the appropriate combination of the geophysical tools and show whether application of an individual method for further investigation of a specific site is beneficial. (C) 2013 Elsevier B.V. All rights reserved.

Genetic diversity of Camellia japonica (Theaceae), a species endangered to East Asia, detected by inter-simple sequence repeat (ISSR)

Abstract Inter-simple sequence repeat (ISSR) markers were used to investigate the genetic diversity within and among thirteen natural populations of Camellia japonica (L.). Twenty ISSR primers gave rise to 211 discernible DNA bands of which 190 (90.1%) were polymorphic. On average each primer gave rise to 10.55 bands including 9.50 bands with polymorphic profile. At the species level, high genetic diversity was detected (PPB: 90.1%; HE: 0.3414; H: 0.5013). However, relatively low genetic diversity existed within populations. Shiko-2 exhibits the greatest level of variability (PPB: 76.8%; HE: 0.2966; H: 0.4319), whereas XS presents its own variability at the lowest level (PPB: 67.3%; HE: 0.2344; H: 0.3478). A relatively high level of genetic differentiation among populations was revealed by Nei's gene diversity statistics (21.3%), Shannon's information measure (21.4%) and analysis of molecular variance (AMOVA) (22.5%). There was significant correlation between genetic distance and geographic distance (r = 0.8154, P

Use of Harriman’s construction in determining molecular equilibrium states

Information-theoretic description of the electron probabilities and currents in molecules is extended to cover the complex amplitudes (wave functions) of quantum mechanics. The classical information measures of Fisher and Shannon, due to the probability/density distributions themselves, are supplemented by the nonclassical terms generated by the wave-function phase or the associated probability current. The previous one-electron development in such an entropic perspective on the molecular electronic structure is extended to cover N-electron states by adopting the Harriman-type framework of equidensity orbitals. This analysis emphasizes the phase part of electronic states, which generates the probability-current density and the associated non-classical entropy contributions, which allow one to distinguish the information content of states generating the same electron density and differing in their current composition. A complementary character of the Fisher and Shannon information measures is explored in the associated vertical (density-constrained) information principles, for determining the equilibrium state corresponding to the fixed ground-state electron density. It is argued that the lowest “thermodynamic” state generally differs from the true ground state of the system, by exhibiting the space-dependent phase and hence also the non-vanishing probability current, linked to the system electron distribution.

Entropic representation in the theory of molecular electronic structure

The entropic perspective on the molecular electronic structure is investigated. Information-theoretic description of electron probabilities is extended to cover the complex amplitudes (wave functions) of quantum mechanics. This analysis emphasizes the entropic concepts due to the phase part of electronic states, which generates the probability current density, thus allowing one to distinguish the information content of states generating the same electron density and differing in their current densities. The classical information measures of Fisher and Shannon, due to the probability/density distributions themselves, are supplemented by the nonclassical terms generated by the wave-function phase or the associated probability current. A complementary character of the Fisher and Shannon information measures is explored and the relationship between these classical information densities is derived. It is postulated to characterize also their nonclassical (phase/current-dependent) contributions. The continuity equations of the generalized information densities are examined and the associated nonclassical information sources are identified. The variational rules involving the quantum-generalized Shannon entropy, which generate the stationary and time-dependent Schrodinger equations from the relevant maximum entropy principles, are discussed and their implications for the system “thermodynamic” equilibrium states are examined. It is demonstrated that the lowest, stationary “thermodynamic” state differs from the true ground state of the system, by exhibiting the space-dependent phase, linked to the modulus part of the wave function, and hence also a nonvanishing probability current.

A mixed-entropic uncertainty relation

We highlight the advantages of using simultaneously the Shannon and Fisher information measures in providing a useful form of the uncertainty relation for the position-momentum case. It does not require any Fourier transformation. The sensitivity is also noteworthy.

Variational principle underlying scale invariant social systems

MaxEnt's variational principle, in conjunction with Shannon's logarithmic information measure, yields only exponential functional forms in straightforward fashion. In this communication we show how to overcome this limitation via the incorporation, into the variational process, of suitable dynamical information. As a consequence, we are able to formulate a somewhat generalized Shannonian Maximum Entropy approach which provides a unifying "thermodynamic-like" explanation for the scale-invariant phenomena observed in social contexts, as city-population distributions. We confirm the MaxEnt predictions by means of numerical experiments with random walkers, and compare them with some empirical data.

On the convergence of Shannon differential entropy, and its connections with density and entropy estimation

This work extends the study of convergence properties of the Shannon differential entropy, and its connections with the convergence of probability measures in the sense of total variation and direct and reverse information divergence. The results relate the topics of distribution (density) estimation, and Shannon information measures estimation, with special focus on the case of differential entropy. On the application side, this work presents an explicit analysis of the density estimation, and differential entropy estimation, for distributions defined on a finite-dimension Euclidean space (Rd,B(Rd)). New consistency results are derived for several histogram-based estimators: the classical product scheme, the Barron's estimator, one of the approaches proposed by Györfi and Van der Meulen, and the data-driven partition scheme of Lugosi and Nobel.

Assessing the monitorability of CO2 saturation in subsurface saline aquifers

We propose a method to assess the geophysical monitorability of petrophysical parameters (in this case CO2 saturation) in subsurface geological reservoirs. The approach is based on measuring and inverting up to six geophysical parameters: P- and S-wave impedances and quality factors (1/attenuation), density and electrical resistivity. We use Shannon's information measure to quantify the information obtained from the inversion of different combinations of geophysical parameters. We thus develop a generic approach to assess the joint monitorability of supercritical CO2 saturation when CO2 is stored in subsurface reservoirs. Uncertainty analysis shows that expected information is in general nonlinearly related to the level of a priori or measurement uncertainty in the geophysical and petrophysical parameters, to the true CO2 saturation, and also to the measurement frequency in the case of seismic measurements. Prior uncertainties in petrophysical parameters such as porosity have a significant effect on the monitorability, highlighting the need for accurate benchmark (pre-injection) measurements and reservoir characterization in the case of time-lapse (4D) monitoring. We show that estimates of P-wave seismic attenuation contribute most to the overall information obtained, and that in principle the joint application of different geophysical methods can significantly improve monitorability. We applied the approach to assess the monitorability of the CO2 saturation after 10 years of simulated injection into a saline aquifer reservoir in the near-shore UK North Sea. Application of the approach to design optimal data combinations for hydrocarbon saturation assessment and monitoring is straightforward.

Genetic variability of the narrow endemic Rhamnus persicifolia Moris (Rhamnaceae) and its implications for conservation

Rhamnus persicifolia Moris is an endemic small tree belonging to the Rhamnus cathartica group, growing along mountainous streams of Central-Eastern Sardinia (Italy). ISSR markers were used to detect the genetic diversity within and among six populations representative of the species distribution range. In spite of the limited distribution of this endemic taxon, fairly high levels of genetic diversity were detected. Percentage of polymorphic bands (PPB), gene diversity ( H S and H T ) and Shannon information measure (Sh) were calculated both at population (PPB = 30.70%, H S = 0.1105, Sh = 0.1646) and at species level (PPB = 68.42%, H T = 0.2066, Sh = 0.3139). The existence of a spatial distribution of genetic diversity in R. persicifolia was revealed by a low gene flow, a fairly high level of genetic differentiation ( G ST = 0.4583) among populations and a positive correlation between genetic and geographic distances (Mantel test, r = 0.71, p = 0.016). The spatial genetic structure was also confirmed with BAPS analysis. Our results show that a certain level of isolation by distance and sex-ratio bias may explain the distribution of genetic diversity among populations. Conservation measures are suggested on the basis of the genetic diversity detected, by implementing an integrated in situ and ex situ conservation program for each population, in order to ensure effective protection for this endemic species. ► Rhamnus persicifolia an endemic tree growing in Sardinia (Italy) was studied. ► ISSR detected the genetic diversity within and among populations of the species. ► Quite high levels of genetic diversity were detected at both levels. ► A spatial genetic structure was detected with BAPS analysis. ► Measures are suggested for integrated in situ and ex situ conservation programs.

Fisher Information at the Edge of Chaos in Random Boolean Networks

We study the order-chaos phase transition in random Boolean networks (RBNs), which have been used as models of gene regulatory networks. In particular we seek to characterize the phase diagram in information-theoretic terms, focusing on the effect of the control parameters (activity level and connectivity). Fisher information, which measures how much system dynamics can reveal about the control parameters, offers a natural interpretation of the phase diagram in RBNs. We report that this measure is maximized near the order-chaos phase transitions in RBNs, since this is the region where the system is most sensitive to its parameters. Furthermore, we use this study of RBNs to clarify the relationship between Shannon and Fisher information measures.

Assessing terrorist risk and FDI using relative information measures

Recent research has explored the determinants of terrorism and the impact that terrorist activity has on a country. However, to date, no research has examined what specific aspect of terrorist risk most significantly impacts a country's ability to operate in the global market. Using the five sub-indices of the World Market Research Center's Global Terrorism Index, this study explores how the different dimensions of terrorist risk impact the inflow of Foreign Direct Investment (FDI) using a Shannon information measure of relationship. The results of this research suggest that a country's proven ability to prevent terrorist activity has the most significant impact on investment decisions in regard to FDI.

On the Discontinuity of the Shannon Information Measures

The Shannon information measures are well known to be continuous functions of the probability distribution for a given finite alphabet. In this paper, however, we show that these measures are discontinuous with respect to almost all commonly used distance measures when the alphabet is countably infinite. Such distance measures include the Kullback-Leibler divergence and the variational distance. Specifically, we show that all the Shannon information measures are in fact discontinuous at all probability distributions. The proofs are based on a probability distribution which can be realized by a discrete-time Markov chain with countably infinite number of states. Our findings reveal that the limiting probability distribution may not fully characterize the asymptotic behavior of a Markov chain. These results explain why certain existing information-theoretical tools are restricted to finite alphabets, and provide hints on how these tools can be extended to countably infinite alphabet.

A Preliminary Study on the Genetic Diversity of the Critically EndangeredCentaures nivea(Asteraceae)

Centaurea nivea (Asteraceae) is endemic to Turkey and distributed only in the Eskişehir Province. It is under risk of extinction due to anthropogenic disturbance, so conservation strategies must be developed urgently. In this study, RAPD technique was used to estimate the level of genetic diversity of C. nivea. Twenty RAPD primers generated 234 bands, of which 215 were polymorphic (91.88%). A high level of genetic diversity was detected both at population (PPB = 72.90%, I = 0.3790, Hs = 0.2527) and at species level (PPB = 91.88%, I = 0.4510, Ht = 0.2963). A moderate level of genetic differentiation among populations was also revealed by Nei's gene diversity analysis (14.73%), Shannon's information measure (15.96%) and AMOVA (7.97%). Patterns of genetic variation among the populations of C. nivea may indicate that the closely located and fragmented populations of C. nivea were likely derived from a previously large population.

Characterization of the Chandrasekhar correlated two-electron wavefunction using Fisher, Shannon and statistical complexity information measures

The three-parameter two-electron correlated analytic wavefunction of Chandrasekhar, parametrized using three physically meaningful conditions on the electron density, is assessed using several information theory measures, the Fisher information, Shannon entropy, and statistical complexity, and compared to the results for the same measures using hydrogenic and Hartree–Fock (HF) orbitals.

Information dynamics: temporal behavior of uncertainty measures

Abstract We carry out a systematic study of uncertainty measures that are generic to dynamical processes of varied origins, provided they induce suitable continuous probability distributions. The major technical tools are the information theory methods and inequalities satisfied by Fisher and Shannon information measures. We focus on the compatibility of these inequalities with the prescribed (deterministic, random or quantum) temporal behavior of pertinent probability densities.

Statistical complexity and Fisher–Shannon information measure of H +2

Using the simple Coulson molecular wave function for the ground state H + 2 molecule as a function of internuclear distance, the first molecular calculations of (a) the statistical complexity measure due to López-Ruiz, Mancini, and Calbet and (b) the Fisher–Shannon information measure in the position and momentum spaces are reported. It is found that the two measures exhibit the molecular bonding in the form of a characteristic minimum around the equilibrium internuclear distance in the product space.

Unique additive information measures—Boltzmann–Gibbs–Shannon, Fisher and beyond

It is proved that the only additive and isotropic information measure that can depend on the probability distribution and also on its first derivative is a linear combination of the Boltzmann–Gibbs–Shannon and Fisher information measures. Power-law equilibrium distributions are found as a result of the interaction of the two terms. The case of second order derivative dependence is investigated and a corresponding additive information measure is given.