Shannon's Entropy Power Research Articles

Investigating the Impact of Xylella Fastidiosa on Olive Trees by the Analysis of MODIS Terra Satellite Evapotranspiration Time Series by Using the Fisher Information Measure and the Shannon Entropy: A Case Study in Southern Italy

Xylella Fastidiosa has been recently detected for the first time in southern Italy, representing a very dangerous phytobacterium capable of inducing severe diseases in many plants. In particular, the disease induced in olive trees is called olive quick decline syndrome (OQDS), which provokes the rapid desiccation and, ultimately, death of the infected plants. In this paper, we analyse about two thousands pixels of MODIS satellite evapotranspiration time series, covering infected and uninfected olive groves in southern Italy. Our aim is the identification of Xylella Fastidiosa-linked patterns in the statistical features of evapotranspiration data. The adopted methodology is the well-known Fisher–Shannon analysis that allows one to characterize the time dynamics of complex time series by means of two informational quantities, the Fisher information measure (FIM) and the Shannon entropy power (SEP). On average, the evapotranspiration of Xylella Fastidiosa-infected sites is characterized by a larger SEP and lower FIM compared to uninfected sites. The analysis of the receiver operating characteristic curve suggests that SEP and FIM can be considered binary classifiers with good discrimination performance that, moreover, improves if the yearly cycle, very likely linked with the meteo-climatic variability of the investigated areas, is removed from the data. Furthermore, it indicated that FIM exhibits superior effectiveness compared to SEP in discerning healthy and infected pixels.

Urban and Peri-Urban Vegetation Monitoring Using Satellite MODIS NDVI Time Series, Singular Spectrum Analysis, and Fisher–Shannon Statistical Method

The purpose of this work was to evaluate the potential of Singular Spectrum Analysis (SSA) and the Fisher–Shannon method to analyse NDVI MODIS time series and to capture and estimate inner vegetation anomalies in forest covers. In particular, the Fisher–Shannon method allows to calculate two quantities, the Fisher Information Measure (FIM) and the Shannon entropy power (SEP), which are used to characterise the complexity of a time series in terms of organisation/disorder. Pilot sites located both in urban (Milano, Torino, and Roma) and peri-urban areas (Appia Park, Castel Porziano, and Castel Volturno) were selected. Among the six sites, Roma, Castel Porziano, and Castel Volturno are affected by the parasite Toumeyella parvicornis. The time series was analysed using the products available in Google Earth Engine. To explore and characterise long-term vegetation dynamics, the time series was analysed using a multistep processing chain based on the (i) normalisation of the satellite time series, (ii) removal of seasonality and any other periodical cycles using SSA, (iii) analysis of the de-trended data using the Fisher–Shannon statistical method, and (iv) validation through comparison with independent data and ancillary information. Our findings point out to a clear discrimination between healthy and unhealthy sites, being the first (Milano, Torino, Appia) characterised by a larger FIM (lower SEP) and the second (Roma, Castel Porziano, Castel Volturno) by a lower FIM (larger SEP). The results of the investigations showed that the use of the SSA and Fisher–Shannon statistical methods coupled with the NDVI time series of the MODIS satellite made it possible to effectively identify and characterise subtle but physically significant signals veiled by seasonality and annual cycles.

Space-time analysis of informational properties of GPS time series recorded at the Campi Flegrei caldera (Italy)

In this work the time dynamics of GPS time series recorded at the Campi Flegrei caldera from 2000 to 2019 was investigated by using the Fisher Information Measure (FIM) and the Shannon entropy power (SEP), two informational methods that allow the detection of changes in the dynamical behavior of a complex system, like the volcanic one, quantifying respectively the order/organization and the disorder/uncertainty. By jointly analyzing the FIM and the SEP through the Fisher-Shannon Information Plane (FSIP) it was shown that the SEP discriminates quite well the vertical displacement time series, while the FIM discriminates better the horizontal ones, which are featured by a more complex behavior. Globally, the most striking change in the SEP is found in 2012, while at local scale abrupt loss of order (indicated by low values of FIM) well correlate with the occurrence of seismic swarms. A further informational quantity, the complexity C (given by the product between SEP and FIM), has been revealed to be sensitive to the baseline deformation level increases and to the small-amplitude modulation of deformation signals within the caldera.

Classification of EEG Signals using Nonlinear Features and Preprocessing Techniques

In this paper we study the effect of nonlinear preprocessing techniques in the classification of electroencephalogram (EEG) signals. These methods are used for classifying the EEG signals captured from epileptic seizure activity and brain tumor category. For the first category, preprocessing is carried out using elliptical filters, and statistical features such as Shannon entropy, mean, standard deviation, skewness and band power. K-Nearest Neighbor (KNN) and Support Vector Machine (SVM) were used for the classification. For the brain tumor EEG signals, empirical mode decomposition is used as a pre-processing technique along with standard statistical features for the classification of normal and abnormal EEG signals. For epileptic signals we have achieved an average accuracy of 94% for a three-class classification and for brain tumor signals we have achieved a classification accuracy of 98% considering it as a two class problem.

Fisher Information and Uncertainty Principle for Skew-Gaussian Random Variables

Fisher information is a measure to quantify information and estimate system-defining parameters. The scaling and uncertainty properties of this measure, linked with Shannon entropy, are useful to characterize signals through the Fisher–Shannon plane. In addition, several non-gaussian distributions have been exemplified, given that assuming gaussianity in evolving systems is unrealistic, and the derivation of distributions that addressed asymmetry and heavy–tails is more suitable. The latter has motivated studying Fisher information and the uncertainty principle for skew-gaussian random variables for this paper. We describe the skew-gaussian distribution effect on uncertainty principle, from which the Fisher information, the Shannon entropy power, and the Fisher divergence are derived. Results indicate that flexibility of skew-gaussian distribution with a shape parameter allows deriving explicit expressions of these measures and define a new Fisher–Shannon information plane. Performance of the proposed methodology is illustrated by numerical results and applications to condition factor time series.

Calculation of information entropies for the 1s2 state of helium‐like ions

AbstractThis work presents analytical and numerical results for the position‐ and momentum‐space information entropies of the 1s2 state of helium‐like ions using different interaction potentials. The potentials that we used are the Yukawa potential and the exponential‐cosine‐screened Coulomb potential. The investigated studies allow us to relate the position‐space information with the momentum‐space information of Shannon and Fisher, as well as Shannon entropy power and the Fisher‐Shannon information product, through different famous relations. The calculation is performed using the one‐electron charge density of entangled two‐parameter wavefunction. On one hand, the results that are presented for 10 members in the helium isoelectronic sequence demonstrate with precision the effect of correlation on bare charge distributions. On the other hand, it leads to some very important results for both the correlated and uncorrelated values of the informatic entropies. Analytical formulas for the momentum‐space information entropies are given. The effect of the nuclear charge and the screening parameter on the information expressions has been studied for both potentials. Detailed computational and numerical values and characteristics of these information quantities, as a function of the screening parameter, are reported here for the first time. New inequality has been proposed with Fisher's total value to measure the correlation of two electrons.

Fisher Shannon analysis of drought/wetness episodes along a rainfall gradient in Northeast Brazil

AbstractWe applied the informational approach of the Fisher‐Shannon method to study the time series of Standardized Precipitation Index (SPI) derived from monthly rainfall data at 133 gauging stations in Northeast (NE) Brazil. The climate in the study region ranges from tropical humid in the coastal area Zona de Mata to tropical semiarid inland Sertão, passing through a sub‐humid transition region Agreste. The time span is from 1962 to 2012, and SPI was calculated for 1, 3, 6, and 12 ‐ month time scales. We calculate information quantifiers Shannon entropy power (SEP) and Fisher information measure (FIM) that quantify signal disorder and structural organization, respectively, and define its position in the Fisher Shannon plane (FS). We find that the SPI‐1 series that describes short term dry/wet conditions display a higher degree of organization (higher FIM) and a lower degree of disorder (lower SEP) than SPI's that describe medium and long term conditions (SPI‐3, SPI‐6, and SPI‐12). By analysing the spatial distribution of information quantifiers, we find that short term dry/wet conditions (SPI‐1) show the lowest disorder and highest organization degree in the deep inland Sertão region and highest disorder and lowest organization degree in the coastal Zona de Mata region, while long term conditions (SPI‐12) show the opposite trend. For medium‐term conditions (SPI‐3 and SPI‐6), the SEP is highest and FIM lowest in the transition sub‐humid Agreste region. Our findings indicate that information theory‐based methods, particularly FS have the potential to discriminate among different climatic regimes.

Informational analysis of Langevin equation of friction in earthquake rupture processes.

In this paper, we analyze the informational properties of time series of slip velocity generated by the Langevin equation of friction in two different frictional regimes: viscous and Coulombian. Representing the generated time series in the Fisher-Shannon plane (where the coordinate axes are the Fisher Information Measure and the Shannon entropy power), the two different frictional regimes are well discriminated. In particular, the viscous regime is characterized by smaller Shannon entropy than the Coulombian one. Furthermore, also the Fisher Information Measure of the slip velocity smoothed by average filter depends on the frictional mechanism, being larger for the viscous regime.

Investigating the time dynamics of wind speed in complex terrains by using the Fisher–Shannon method

In this paper, the time dynamics of the daily means of wind speed measured in complex mountainous regions are investigated. For 293 measuring stations distributed over all Switzerland, the Fisher information measure and the Shannon entropy power are calculated. The results reveal a clear relationship between the computed measures and both the elevation of the wind stations and the slope of the measuring sites. In particular, the Shannon entropy power and the Fisher information measure have their highest (respectively lowest) values in the Alps, where the time dynamics of wind speed follows a more disordered pattern. The spatial mapping of the calculated quantities allows the identification of two regions, which is in agreement with the topography of the Swiss territory. The present study could contribute to a better characterization of the temporal dynamics of wind speed in complex mountainous terrain.

Fisher-Shannon Complexity Analysis of High-Frequency Urban Wind Speed Time Series.

One-hertz wind time series recorded at different levels (from 1.5–25.5 m) in an urban area are investigated by using the Fisher–Shannon (FS) analysis. FS analysis is a well-known method to gain insight into the complex behavior of nonlinear systems, by quantifying the order/disorder properties of time series. Our findings reveal that the FS complexity, defined as the product between the Fisher information measure and the Shannon entropy power, decreases with the height of the anemometer from the ground, suggesting a height-dependent variability in the order/disorder features of the high-frequency wind speed measured in urban layouts. Furthermore, the correlation between the FS complexity of wind speed and the daily variance of the ambient temperature shows a similar decrease with the height of the wind sensor. Such correlation is larger for the lower anemometers, indicating that ambient temperature is an important forcing of the wind speed variability in the vicinity of the ground.

On Generalized Stam Inequalities and Fisher–Rényi Complexity Measures

Information-theoretic inequalities play a fundamental role in numerous scientific and technological areas (e.g., estimation and communication theories, signal and information processing, quantum physics, …) as they generally express the impossibility to have a complete description of a system via a finite number of information measures. In particular, they gave rise to the design of various quantifiers (statistical complexity measures) of the internal complexity of a (quantum) system. In this paper, we introduce a three-parametric Fisher–Rényi complexity, named ( p , β , λ ) -Fisher–Rényi complexity, based on both a two-parametic extension of the Fisher information and the Rényi entropies of a probability density function ρ characteristic of the system. This complexity measure quantifies the combined balance of the spreading and the gradient contents of ρ , and has the three main properties of a statistical complexity: the invariance under translation and scaling transformations, and a universal bounding from below. The latter is proved by generalizing the Stam inequality, which lowerbounds the product of the Shannon entropy power and the Fisher information of a probability density function. An extension of this inequality was already proposed by Bercher and Lutwak, a particular case of the general one, where the three parameters are linked, allowing to determine the sharp lower bound and the associated probability density with minimal complexity. Using the notion of differential-escort deformation, we are able to determine the sharp bound of the complexity measure even when the three parameters are decoupled (in a certain range). We determine as well the distribution that saturates the inequality: the ( p , β , λ ) -Gaussian distribution, which involves an inverse incomplete beta function. Finally, the complexity measure is calculated for various quantum-mechanical states of the harmonic and hydrogenic systems, which are the two main prototypes of physical systems subject to a central potential.

A systematic semiempirical study of information inequalities for the vibrational levels of a diatomic molecule for the example of the ground electronic state of 7Li2

A systematic study of the dependences of information inequalities relating the variance, Fisher information, and Shannon entropy power in the coordinate and momentum spaces (the Cramer–Rao, Stam, Bialynicki–Birula–Mycielsky–Beckner, and other inequalities) on the vibrational quantum number of a diatomic molecule is performed for the first time. These dependences are calculated for the ground electronic state of the 7Li2 molecule using the semiempirical data available in the literature on the many-parameter potential curve of this state. Specific features of these dependences have been analyzed, their essentially nonmonotonic behavior was revealed, and not only quantitative, but also qualitative, distinctions between these dependences were observed. These results were compared with the results of our calculation in the classical mechanics approximation, and significant (up to a factor of 3.5) discrepancies between these data were observed not only for low-lying, but also for highly excited vibrational levels that are close to the dissociation limit. The results obtained can be used in quantum informatics, analysis of an intramolecular structure and the interaction processes involving the vibrational states of diatomic molecules.

Classification of Human Emotions using Multiwavelet Transform based Features and Random Forest Technique

Objective: To extract statistical features from EEG signal for human emotion recognition. Methods: Emotions, the “inner” state of a person play a vital role in analysing the state of mind. In this paper, a new method for human emotion recognition using Multi Wavelet Transform (MWT) and random forest (ensemble technique) for classification of human emotions from EEG signals has been proposed. The experiment is conducted on EEG database for emotions i.e., DEAP data set. In present work, the data set used contains EEG recording from 4 participants (S01, S02, S03, S04) from 15 channel (FP1, FP2, F3, P3, F4, T7, T8, P4, O1, PZ, PO3, O2, P7, CP2, and C4) having 40 trails each. The paper explore the capability of proposed features namely Mean, Standard deviation, Variance, Shannon entropy, Hjorth parameters and Band power. Findings: The feature set obtained using Multi-Wavelet are then used as input for MLP, KNN, MC-SVM with Puk kernel function and Random Forest (ensemble) classifier for the classification of emotions. The classification accuracy for different emotion state happy 99.8%, sad 98.3%, exciting 95%, hate 96.4% are obtained by the proposed method. Overall 98.1% for classification of emotions (happy, sad, exciting, hate) from EEG signals. Application: The findings are helpful in monitoring alertness, cognitive engagement of patients with impaired motor functions; recognition of emotions; investigating sleep disorders and physiology. In future, efficient feature extraction algorithm using different multiwavelet functions and with a different set of features which include the temporal dynamics of brain signals in the human cognitive system can be considered.

Discriminating Between Different Streamflow Regimes by Using the Fisher-Shan Method: An Application to the Colombia Rivers

The Fisher-Shannon (FS) information plane, defined by the Fisher information measure (FIM) and the Shannon entropy power (NX), was robustly used to investigate the complex dynamics of eight monthly streamflow time series in Colombia. In the FS plane the streamflow series seem to aggregate into two different clusters corresponding to two different climatological regimes in Colombia. Our findings suggest the use of the statistical quantity defined by the FS information plane as a tool to discriminate among different hydrological regimes.

The Entropy Power Inequality and Mrs. Gerber's Lemma for Groups of Order <inline-formula> <tex-math notation="TeX">${2^{n}}$ </tex-math></inline-formula>

Shannon's entropy power inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key role in resolving a number of problems in information theory. It is therefore interesting to examine the existence of a similar inequality for discrete random variables. In this paper, we obtain an entropy power inequality for random variables taking values in a group of order 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> , i.e., for such a group G, we explicitly characterize the function f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</sub> (x, y) giving the minimum entropy of the sum of two independent G-valued random variables with respective entropies x and y. Random variables achieving the extremum in this inequality are thus the analogs of Gaussians in this case, and these are also determined. It turns out that f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</sub> (x, y) is convex in x for fixed y and, by symmetry, convex in y for fixed x. This is a generalization to groups of order 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> of the result known as Mrs. Gerber's Lemma.

Informational analysis of apparent Earth’s resistivity time series to assess the reliability of magnetotelluric measurements

We used the informational methods of the Shannon entropy power and the Fisher Information Measure to analyze the time dynamics of one year of semi-daily values of apparent resistivity, calculated on the base of the magnetotelluric (MT) monitoring at Penghu Island, Taiwan. The performed statistics indicate that the day-time measurements present higher disorder and lower organization than the night-time measurements. Such difference is more evident for the high-frequency band of resistivity data, which suggests that possibly some source of the electromagnetic field could be of anthropic origin. Furthermore, we also note the characteristics of so-called MT dead-band with the periodicity of ∼10 s could be revealed in the high values of Shannon entropy power.

Fisher–Shannon analysis of seismograms of tsunamigenic and non-tsunamigenic earthquakes

Recognizing the tsunamigenic potential of an earthquake presents a challenge in the context of studies devoted to early warnings of tsunami events. In this paper, we show the feasibility of discriminating between tsunamigenic and non-tsunamigenic earthquakes by analyzing seismograms on the basis of the Fisher–Shannon method, which is used to analyze the order/organization structure of a complex and nonstationary time series. The results obtained results show that by combining the measures of the Shannon entropy power and the Fisher information measure, the discrimination between the two groups—tsunamigenic and non-tsunamigenic earthquakes—is very efficiently achieved. These results could contribute to the assessment of tsunami early warning systems.

On MMSE Crossing Properties and Implications in Parallel Vector Gaussian Channels

The scalar additive Gaussian noise channel has the “single crossing point” property between the minimum mean square error (MMSE) in the estimation of the input given the channel output, assuming a Gaussian input to the channel, and the MMSE assuming an arbitrary input. This paper extends the result to the parallel vector additive Gaussian channel in three phases. The channel matrix is the identity matrix, and we limit the Gaussian input to a vector of Gaussian i.i.d. elements. The “single crossing point” property is with respect to the signal-to-noise ratio (as in the scalar case). The channel matrix is arbitrary, and the Gaussian input is limited to an independent Gaussian input. A “single crossing point” property is derived for each diagonal element of the MMSE matrix. The Gaussian input is allowed to be an arbitrary Gaussian random vector. A “single crossing point” property is derived for each eigenvalue of the difference matrix between the two MMSE matrices. These three extensions are then translated to new information theoretic properties on the mutual information, using the I-MMSE relationship, a fundamental relationship between estimation theory and information theory revealed by Guo and coworkers. The results of the last phase are also translated to a new property of Fisher information. Finally, the applicability of all three extensions on information theoretic problems is demonstrated through a proof of a special case of Shannon's vector entropy power inequality, a converse proof of the capacity region of the parallel degraded broadcast channel (BC) under an input per-antenna power constraint and under an input covariance constraint, and a converse proof of the capacity region of the compound parallel degraded BC under an input covariance constraint.

Investigating the time dynamics of monthly rainfall time series observed in northern Lebanon by means of the detrended fluctuation analysis and the Fisher-Shannon method

We investigate the time dynamics of monthly rainfall series intermittently recorded on seven climatic stations in northern Lebanon from 1939 to 2010 using the detrending fluctuation analysis (DFA) and the Fisher-Shannon (FS) method. The DFA is employed to study the scaling properties of the series, while the FS method to analyze their order/organization structure. The obtained results indicate that most all the stations show a significant persistent behavior, suggesting that the dynamics of the rainfall series is governed by positive feedback mechanisms. Furthermore, we found that the Fisher Information Measure (the Shannon entropy power) seems to decrease (increase) with the height of the rain gauge; this indicates that the rainfall series appear less organized and less regular for higher-located stations. Such findings could be useful for a better comprehension of the climatic regimes governing northern Lebanon.

The Concavity of Rényi Entropy Power

We associate to the p-th R\'enyi entropy a definition of entropy power, which is the natural extension of Shannon's entropy power and exhibits a nice behaviour along solutions to the p-nonlinear heat equation in $R^n$. We show that the R\'enyi entropy power of general probability densities solving such equations is always a concave function of time, whereas it has a linear behaviour in correspondence to the Barenblatt source-type solutions. We then shown that the p-th R\'enyi entropy power of a probability density which solves the nonlinear diffusion of order p, is a concave function of time. This result extends Costa's concavity inequality for Shannon's entropy power to R\'enyi entropies.