Fisher Information Measure Research Articles

A Unified Approach to Cramér–Rao Inequalities

A unified approach is presented for establishing a broad class of Cramér–Rao inequalities for the location parameter, including, as special cases, the original inequality of Cramér and Rao, as well as an <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L^{p}$</tex></formula> version recently established by the authors. The new approach allows for generalized moments and Fisher information measures to be defined by convex functions that are not necessarily homogeneous. In particular, it is shown that associated with any log-concave random variable whose density satisfies certain boundary conditions is a Cramér–Rao inequality for which the given log-concave random variable is the extremal. Applications to specific instances are also provided.

Informational analysis of apparent Earth’s resistivity time series to assess the reliability of magnetotelluric measurements

We used the informational methods of the Shannon entropy power and the Fisher Information Measure to analyze the time dynamics of one year of semi-daily values of apparent resistivity, calculated on the base of the magnetotelluric (MT) monitoring at Penghu Island, Taiwan. The performed statistics indicate that the day-time measurements present higher disorder and lower organization than the night-time measurements. Such difference is more evident for the high-frequency band of resistivity data, which suggests that possibly some source of the electromagnetic field could be of anthropic origin. Furthermore, we also note the characteristics of so-called MT dead-band with the periodicity of ∼10 s could be revealed in the high values of Shannon entropy power.

Irregularity analysis of CO, NO2 and O3 concentrations at traffic, commercial and low activity sites in Delhi

The irregularity analysis of exceedance time series of gaseous pollutants CO, NO2 and O3 is carried out using Shannon entropy and Fisher information measure. The data observed during 2007–2010 at three sites with different land-use activities in Delhi are analyzed. CO and NO2 showed irregular behavior at both, low anthropogenic activity and commercial activity sites, whereas at traffic site both the pollutant concentrations showed regular behavior. The irregularity is attributed to the multiplicity in emission sources at low activity and commercial site and regular behavior is observed due to the uniformity and well defined source characteristics at the traffic site. O3 at three sites showed irregular behavior owing to its secondary nature. Fisher–Shannon information plane showed the grouping of three pollutants except CO and NO2 at traffic and O3 at low activity site suggesting the similar temporal characteristics of the pollutants even at the sites with different land-use activities.

Singular spectrum analysis and Fisher–Shannon analysis of spring flow time series: An application to Anjar Spring, Lebanon

In this study, the time dynamics of water flow from Anjar Spring was investigated, which is one of the major issuing springs in the central part of Lebanon. Likewise, many water sources in Lebanon, this spring has no continuous records for the discharge, and this would prevent the application of standard time series analysis tools. Furthermore, the highly nonstationary character of the series implies that suited methodologies can be employed to get insight into its dynamical features. Therefore, the Singular Spectrum Analysis (SSA) and Fisher–Shannon (FS) method, which are useful methods to disclose dynamical features in noisy nonstationary time series with gaps, are jointly applied to analyze the Anjar Spring water flow series. The SSA revealed that the series can be considered as the superposition of meteo-climatic periodic components, low-frequency trend and noise-like high-frequency fluctuations. The FS method allowed to extract and to identify among all the SSA reconstructed components the long-term trend of the series. The long-term trend is characterized by higher Fisher Information Measure (FIM) and lower Shannon entropy, and thus, represents the main informative component of the whole series. Generally water discharge time series presents very complex time structure, therefore the joint application of the SSA and the FS method would be very useful in disclosing the main informative part of such kind of data series in the view of existing climatic variability and/or anthropogenic challenges.

A novel statistical approach for detection of suspicious regions in digital mammogram

In this paper, we propose a novel algorithm to detect the suspicious regions on digital mammograms that based on the Fisher information measure. The proposed algorithm is tested different types and categories of mammograms (fatty, fatty-glandular and dense glandular) within mini-MIAS database (Mammogram Image Analysis Society database (UK)). The proposed method is compared with a different segmentation based information theoretical methods to demonstrate their effectiveness. The experimental results on mammography images showed the effectiveness in the detection of suspicious regions. This study can be a part of developing a computer-aided decision (CAD) system for early detection of breast cancer.

Fisher–Shannon analysis of seismograms of tsunamigenic and non-tsunamigenic earthquakes

Recognizing the tsunamigenic potential of an earthquake presents a challenge in the context of studies devoted to early warnings of tsunami events. In this paper, we show the feasibility of discriminating between tsunamigenic and non-tsunamigenic earthquakes by analyzing seismograms on the basis of the Fisher–Shannon method, which is used to analyze the order/organization structure of a complex and nonstationary time series. The results obtained results show that by combining the measures of the Shannon entropy power and the Fisher information measure, the discrimination between the two groups—tsunamigenic and non-tsunamigenic earthquakes—is very efficiently achieved. These results could contribute to the assessment of tsunami early warning systems.

Defining statistical perceptions with an empirical Bayesian approach

Extracting statistical structures (including textures or contrasts) from a natural stimulus is a central challenge in both biological and engineering contexts. This study interprets the process of statistical recognition in terms of hyperparameter estimations and free-energy minimization procedures with an empirical Bayesian approach. This mathematical interpretation resulted in a framework for relating physiological insights in animal sensory systems to the functional properties of recognizing stimulus statistics. We applied the present theoretical framework to two typical models of natural images that are encoded by a population of simulated retinal neurons, and demonstrated that the resulting cognitive performances could be quantified with the Fisher information measure. The current enterprise yielded predictions about the properties of human texture perception, suggesting that the perceptual resolution of image statistics depends on visual field angles, internal noise, and neuronal information processing pathways, such as the magnocellular, parvocellular, and koniocellular systems. Furthermore, the two conceptually similar natural-image models were found to yield qualitatively different predictions, striking a note of warning against confusing the two models when describing a natural image.

A Novel Approach for MRI Brain Images Segmentation

Segmentation of brain from magnetic resonance (MR) images has important applications in neuroimaging, in particular it facilitates in extracting different brain tissues such as cerebrospinal fluids, white matter and gray matter. That helps in determining the volume of the tissues in three-dimensional brain MR images, which yields in analyzing many neural disorders such as epilepsy and Alzheimer disease. The Fisher information is a measure of the fluctuations in the observations. In a sense, the Fisher information of an image specifies the quality of the image. In this paper, we developed a new thresholding method using the Fisher information measure and intensity contrast to segment medical images. It is the weighted sum of the Fisher information measure and intensity contrast between the object and background. This technique is a powerful method for noisy image segmentation. The method applied on a normal MR brain images and a glioma MR brain images. Experimental results show that the use of the Fisher information effectively segmented MR brain images.

Investigating the time dynamics of monthly rainfall time series observed in northern Lebanon by means of the detrended fluctuation analysis and the Fisher-Shannon method

We investigate the time dynamics of monthly rainfall series intermittently recorded on seven climatic stations in northern Lebanon from 1939 to 2010 using the detrending fluctuation analysis (DFA) and the Fisher-Shannon (FS) method. The DFA is employed to study the scaling properties of the series, while the FS method to analyze their order/organization structure. The obtained results indicate that most all the stations show a significant persistent behavior, suggesting that the dynamics of the rainfall series is governed by positive feedback mechanisms. Furthermore, we found that the Fisher Information Measure (the Shannon entropy power) seems to decrease (increase) with the height of the rain gauge; this indicates that the rainfall series appear less organized and less regular for higher-located stations. Such findings could be useful for a better comprehension of the climatic regimes governing northern Lebanon.

Use of Harriman’s construction in determining molecular equilibrium states

Information-theoretic description of the electron probabilities and currents in molecules is extended to cover the complex amplitudes (wave functions) of quantum mechanics. The classical information measures of Fisher and Shannon, due to the probability/density distributions themselves, are supplemented by the nonclassical terms generated by the wave-function phase or the associated probability current. The previous one-electron development in such an entropic perspective on the molecular electronic structure is extended to cover N-electron states by adopting the Harriman-type framework of equidensity orbitals. This analysis emphasizes the phase part of electronic states, which generates the probability-current density and the associated non-classical entropy contributions, which allow one to distinguish the information content of states generating the same electron density and differing in their current composition. A complementary character of the Fisher and Shannon information measures is explored in the associated vertical (density-constrained) information principles, for determining the equilibrium state corresponding to the fixed ground-state electron density. It is argued that the lowest “thermodynamic” state generally differs from the true ground state of the system, by exhibiting the space-dependent phase and hence also the non-vanishing probability current, linked to the system electron distribution.

Entropic representation in the theory of molecular electronic structure

The entropic perspective on the molecular electronic structure is investigated. Information-theoretic description of electron probabilities is extended to cover the complex amplitudes (wave functions) of quantum mechanics. This analysis emphasizes the entropic concepts due to the phase part of electronic states, which generates the probability current density, thus allowing one to distinguish the information content of states generating the same electron density and differing in their current densities. The classical information measures of Fisher and Shannon, due to the probability/density distributions themselves, are supplemented by the nonclassical terms generated by the wave-function phase or the associated probability current. A complementary character of the Fisher and Shannon information measures is explored and the relationship between these classical information densities is derived. It is postulated to characterize also their nonclassical (phase/current-dependent) contributions. The continuity equations of the generalized information densities are examined and the associated nonclassical information sources are identified. The variational rules involving the quantum-generalized Shannon entropy, which generate the stationary and time-dependent Schrodinger equations from the relevant maximum entropy principles, are discussed and their implications for the system “thermodynamic” equilibrium states are examined. It is demonstrated that the lowest, stationary “thermodynamic” state differs from the true ground state of the system, by exhibiting the space-dependent phase, linked to the modulus part of the wave function, and hence also a nonvanishing probability current.

A mixed-entropic uncertainty relation

We highlight the advantages of using simultaneously the Shannon and Fisher information measures in providing a useful form of the uncertainty relation for the position-momentum case. It does not require any Fourier transformation. The sensitivity is also noteworthy.

Product of position and momentum Fisher information measures under homogeneous potentials

The Fisher information has proved to be a very useful tool in the informational analysis of atomic and molecular systems. In this paper we show analytically that the net Fisher information measure, defined as the product of Fisher information integrals of the electron density in the position, Ir , and momentum, Ip, spaces generated by λV(r) remains independent of strength of the homogeneous potential, λ. Specifically, IrIp=Ir(λ)Ip(λ). This proof must be viewed in the light of a special characteristic of the homogeneous potentials according to which the scaled densities are the eigendensities of the Hamiltonian of the same form with a modified potential strength constant. Furthermore, the behavior of a new form of Fisher information for eigendensities of homogeneous potentials is explored.

Information-theoretical complexity for the hydrogenic identity S N 2 exchange reaction

We investigate the complexity of the hydrogenic identity S N 2 exchange reaction by means of information-theoretic functionals such as disequilibrium (D), exponential entropy (L), Fisher information (I), power entropy (J) and joint information-theoretic measures, i.e., the I–D, D–L and I–J planes and the Fisher–Shannon (FS) and Lopez-Mancini-Calbet (LMC) shape complexities. The several information-theoretic measures of the one-particle density were computed in position (r) and momentum (p) spaces. The analysis revealed that the chemically significant regions of this reaction can be identified through most of the information-theoretic functionals or planes, not only the ones which are commonly revealed by the energy, such as the reactant/product (R/P) and the transition state (TS), but also those that are not present in the energy profile such as the bond cleavage energy region (BCER), the bond breaking/forming regions (B–B/F) and the charge transfer process (CT). The analysis of the complexities shows that the energy profile of the identity S N 2 exchange reaction bears no simple behavior with respect to the LMC and FS measures. Most of the chemical features of interest (BCER, B–B/F and CT) are only revealed when particular information-theoretic aspects of localizability (L or J), uniformity (D) and disorder (I) are considered.

Parameter-free ansatz for inferring ground state wave functions of even convex potentials

Schrödinger's equation (SE) and the information-optimizing principle based on Fisher's information measure are intimately linked (Frieden et al 1999 Phys. Rev. E 60 48), which entails the existence of a Legendre transform structure underlying the SE (Flego et al 2011 J. Math. Phys.52 082103). In this paper, we show that the existence of such a structure allows, via the virial theorem, for the formulation of a parameter-free ground state's SE ansatz for a rather large family of potentials. The parameter-free nature of the ansatz derives from the structural information it incorporates through its Legendre properties.

Bounds for Fisher information and its production under flow

We prove that two well-known measures of information are interrelated in interesting and useful ways when applied to nonequilibrium circumstances. A nontrivial form of the lower bound for the Fisher information measure is derived in presence of a flux vector, which satisfies the continuity equation. We also establish a novel upper bound on the time derivative (production) in terms of the arrow of time and derive a lower bound by the logarithmic Sobolev inequality. These serve as the revealing dynamics of the information content and its limitations pertaining to nonequilibrium processes.

Fisher Information and Quantum Mechanics

A suggestive relation links Fisher' information measure (FIM)Iand Schrodinger equation (SE). The connection is based upon the fact that the constrained minimization of I leads to a SE. This, in turn, is the origin of intriguing relationships between various aspects of SE, on the one hand, and the formalism of statistical mechanics derived from Jaynes's maximum entropy principle (MaxEnt), on the other one. The link entails the existence of a Legendre transform structure underlying the SE, which allows for the emergence of two first-order differential equations that must, respectively, be satisfied by i) the Fisher measure and ii) the SE energy eigenvalues. The complete A) I-solution and B) energysolution are both obtained bypassing the SE and, furthermore, linked by the Legendre structure.

Informational analysis of seismic sequences by applying the Fisher Information Measure and the Shannon entropy: An application to the 2004–2010 seismicity of Aswan area (Egypt)

The interevent-time (IET) and interevent-distance (IED) series of seismic events occurred at Aswan area (Egypt) from 2004 to 2010 were investigated by means of the Fisher Information Measure and the Shannon entropy. The analysis was performed varying the depth and the magnitude thresholds. The results point out to an increase of level of organization and order with the decrease of magnitude threshold and the increase of depth threshold for the IET series, while the IED series are characterized by a level of uncertainty approximately constant with the threshold magnitude. The complexity measure, calculated as the product of the Fisher Information Measure and the Shannon entropy power, presents very similar pattern for both the types of seismic series, indicating an increasing complexity with the decrease of the threshold magnitude and the increase of the threshold depth.

Maximum Likelihood Estimators for a Supercritical Branching Diffusion Process

The log-likelihood of a nonhomogeneous Branching Diffusion Process under several conditions assuring existence and uniqueness of the diffusion part and nonexplosion of the branching process. Expressions for different Fisher information measures are provided. Using the semimartingale structure of the process and its local characteristics, a Girsanov-type result is applied. Finally, an Ornstein-Uhlenbeck process with finite reproduction mean is studied. Simulation results are discussed showing consistency and asymptotic normality.

Discriminating climatological regimes in rainfall time series by using the Fisher-Shannon method

The Fisher-Shannon (FS) information plane, defined by the Fisher information measure (FIM) and the Shannon entropy power (Nx), was robustly used to investigate the complex dynamics of 8 long monthly rainfall time series in central Argentina, recorded from 1860 to 2006. In the FS plane, the rainfall series seem to aggregate into three different clusters corresponding to three different climatological regimes in central Argentina. Our findings suggest the use of the statistical quantity defined by the FS information plane as a tool to discriminate among different climatological regimes. Key words: Rain, Fisher information measure, Shannon entropy.