Use Of Information Theory Research Articles

Photoacoustic Waveform Design for Optimal Parameter Estimation Based on Maximum Mutual Information

Waveform design is a potentially significant approach to improve the performance of an imaging or detection system. Photoacoustic imaging is a rapidly developing field in recent years; however, photoacoustic waveform design has not been extensively investigated. This paper considers the problem of photoacoustic waveform design for parameter estimation under constraints on input energy. The use of information theory is exploited to formulate and solve this optimal waveform design problem. The approach yields the optimal waveform power spectral density. Direct inverse Fourier transform of the optimal waveform frequency spectrum amplitude is proposed to obtain a real waveform in the time domain. Absorbers are assumed to be stochastic absorber ensembles with uncertain duration and location parameters. Simulation results show the relationship between absorber parameter distribution and the characteristics of optimal waveforms. Comparison of optimal waveforms for estimation, optimal waveforms for detection (signal-to-noise ratio) and other commonly used waveforms are also discussed. The symmetry properties of the forward and inverse Fourier Transforms are used to analyze the time and frequency properties and provide a heuristic view of how different goals affect the choice of waveform.

Analyzing the Effect of the Intra-Pixel Position of Small PSFs for Optimizing the PL of Optical Subpixel Localization

Subpixel localization techniques for estimating the positions of point-like images captured by pixelated image sensors have been widely used in diverse optical measurement fields. With unavoidable imaging noise, there is a precision limit (PL) when estimating the target positions on image sensors, which depends on the detected photon count, noise, point spread function (PSF) radius, and PSF’s intra-pixel position. Previous studies have clearly reported the effects of the first three parameters on the PL but have neglected the intra-pixel position information. Here, we develop a localization PL analysis framework for revealing the effect of the intra-pixel position of small PSFs. To accurately estimate the PL in practical applications, we provide effective PSF (ePSF) modeling approaches and apply the Cramér–Rao lower bound. Based on the characteristics of small PSFs, we first derive simplified equations for finding the best PL and the best intra-pixel region for an arbitrary small PSF; we then verify these equations on real PSFs. Next, we use the typical Gaussian PSF to perform a further analysis and find that the final optimum of the PL is achieved at the pixel boundaries when the Gaussian radius is as small as possible, indicating that the optimum is ultimately limited by light diffraction. Finally, we apply the maximum likelihood method. Its combination with ePSF modeling allows us to successfully reach the PL in experiments, making the above theoretical analysis effective. This work provides a new perspective on combining image sensor position control with PSF engineering to make full use of information theory, thereby paving the way for thoroughly understanding and achieving the final optimum of the PL in optical localization.

Information Length Quantification and Forecasting of Power Systems Kinetic Energy

One of the short-coming challenges of power systems operation and planning is the difficulty to quantify the variability of power systems Kinetic Energy (KE) to unveil online additional information for the system operators’ decisions support. KE monitoring requires innovative methods to analyse the continuous fluctuations in the KE power’s systems. In this paper, we propose the use of information theory, specifically the concept of Information Length (IL), as a way to provide useful insights into the power system KE variability and to demonstrate its utility as a starting point in decision making for power systems management. The proposed IL metric is applied to monthly collected data from the Nordic Power System during three consecutive years in order to investigate the KE evolution. Our results reveal that the proposed method provides an effective description of the seasonal statistical variability enabling the identification of the particular month and day that have the least and the most KE variability. Additionally, by applying a Long Short-Term Memory (LSTM) neural network model to estimate the value of the IL on-line, we also show the possibility of using the metric as data-driven support.

Quantitative Characterization of Complex Systems—An Information Theoretic Approach

A significant increase in System-of-Systems (SoS) is currently observed in the social and technical domains. As a result of the increasing number of constituent system components, Systems of Systems are becoming larger and more complex. Recent research efforts have highlighted the importance of identifying innovative statistical and theoretical approaches for analyzing complex systems to better understand how they work. This paper portrays the use of an agnostic two-stage examination structure for complex systems aimed towards developing an information theory-based approach to analyze complex technical and socio-technical systems. Towards the goal of characterizing system complexity with information entropy, work was carried out in exploring the potential application of entropy to a simulated case study to illustrate its applicability and to establish the use of information theory within the broad horizon of complex systems. Although previous efforts have been made to use entropy for understanding complexity, this paper provides a basic foundation for identifying a framework to characterize complexity, in order to analyze and assess complex systems in different operational domains.

Contribution of each leg in the static postural stability of unilateral transtibial amputees, a study with Information Theory

Introduction: Transtibial amputees tend to isolate themselves because they lack confidence in performing daily tasks; rehabilitation processes such as the use of prostheses are carried out for their social reintegration, where postural stability must be ensured, and activities allowed in diverse contexts. Objective: Verify the contribution of each leg and direction (anterior/posterior and medial/lateral) of the center of pressure (CoP) in the static postural stability of unilateral transtibial amputees due to landmines using information theory. Methodology: A quantitative experimental design was applied to examine the displacement and velocity of the CoP using infometric measurements in nine transtibial amputees, and the non-parametric U Mann-Whitney test was applied to recognize statistically notable differences (p <0.05) between each leg. Results: There are statistically notable differences between the legs in the variables of displacement and velocity. Each leg contributes differently to the control of the static bipedal posture, and their behavior in the directions of movement of the CoP differs from each other, but they work together to generate compensation strategies that allow maintaining stability. Conclusions: The use of information theory made it possible to evaluate stability objectively, without specifying the kind of relationship between this and the input variables, which favored the recognition of the CoP shift as its best discriminant in comparison with velocity. Transtibial amputees control static bipedal posture with the non-amputated leg to compensate for the loss of lower leg afferents and efferents due to the amputation. The results demonstrate the suitability of using information theory in the evaluation of lateralized postural control in pathological conditions.

Complexity entropy-analysis of monthly rainfall time series in northeastern Brazil

In this work we analyze predictability and complexity of monthly rainfall temporal series recorded from 1950 to 2012, at 133 gauging stations in Pernambuco state, northeastern Brazil. To this end we use the complexity entropy causality plane (CECP) and Fisher Shannon plane (FS) formed by information quantifiers permutation entropy, permutation statistical complexity, and Fisher information measure, which serve to evaluate randomness and structural organization of the underlying process. By comparing the locations of analyzed stations in CECP and FS and performing statistical significance test, we distinguish rainfall regime in the deep inland, drier Sertão region, from the intermediate inland Agreste, and coastal, tropical humid Zona da Mata regions. We also perform time dependent CECP and FS analysis and for Sertão region identify the periods of higher entropy (lower complexity and Fisher information measure) which are related to El Niño episodes and historical droughts in northeastern Brazil. Our work represents a contribution to establishing the use of information theory based methods in climatological studies.

First-principles prediction of the information processing capacity of a simple genetic circuit.

Given the stochastic nature of gene expression, genetically identical cells exposed to the same environmental inputs will produce different outputs. This heterogeneity has been hypothesized to have consequences for how cells are able to survive in changing environments. Recent work has explored the use of information theory as a framework to understand the accuracy with which cells can ascertain the state of their surroundings. Yet the predictive power of these approaches is limited and has not been rigorously tested using precision measurements. To that end, we generate a minimal model for a simple genetic circuit in which all parameter values for the model come from independently published data sets. We then predict the information processing capacity of the genetic circuit for a suite of biophysical parameters such as protein copy number and protein-DNA affinity. We compare these parameter-free predictions with an experimental determination of protein expression distributions and the resulting information processing capacity of E. coli cells. We find that our minimal model captures the scaling of the cell-to-cell variability in the data and the inferred information processing capacity of our simple genetic circuit up to a systematic deviation.

Object-based feature extraction for hyperspectral data using firefly algorithm

Object-based classification methods can improve the accuracy of hyperspectral image classification due to the fact that they incorporate spatial information into the classification procedure by assigning neighboring pixels into the same class. In this paper, a new object-based feature extraction method is proposed which makes use of information theory to reduce the Bayes error. In this way, the proposed method exploits higher order statistics for feature extraction which are very effective for non Gaussian data such as hyperspectral images. The criterion to be minimized is composed of three mutual information terms. The first and second terms, consider the maximal relevance and minimal redundancy, respectively, while the third term takes into account the segmentation map containing disjoint spatial regions. To obtain the segmentation map, we apply the firefly clustering algorithm whose fitness function simultaneously considers the intra-distance between samples and their cluster centroids, and inter-distance between centroids of any two clusters. Our experimental results, performed using a variety of hyperspectral scenes, indicate that the proposed framework gives better classification results than some state-of the-art spectral–spatial feature extraction methods.

Informational Model of State Change in a Mechanical System

The use of information theory to formulate an informational model of the change in state of mechanical systems is considered. By that means, models of the change in state of system components may be developed for different operating conditions. In regions of the system’s state space corresponding to critical values, the probability of a fault may be determined.

Assessing the State of Mechanical Systems of Different Complexity

Abstract—The use of information theory to assess the state of mechanical systems of different complexity is considered.

The Boltzmann Constant: Evaluation of Measurement Relative Uncertainty Using the Information Approach

The purpose of this work is to prove that only by applying a theoretically sound information approach to developing a model for measuring the Boltzmann constant, one can justify and calculate the value of the required relative uncertainty. A dimensionless parameter (comparative uncertainty) was proposed as a universal metric for comparing experimental measurements of Boltzmann constant and simulated data. Examples are given of applying the proposed original method for calculating the relative uncertainty in measuring the Boltzmann constant using an acoustic gas thermometer, dielectric constant gas thermometer, Johnson noise thermometer, Doppler broadening thermometer. The proposed approach is theoretically justified and devoid of the shortcomings inherent in the CODATA concept: a statistically significant trend, a cumulative value of consensus or a statistical control. We tried to show how a mathematical-expert formalism can be replaced by a simple, theoretically grounded postulate on the use of information theory in measurements.

Information Theory and Cognition: A Review.

We examine how information theory has been used to study cognition over the last seven decades. After an initial burst of activity in the 1950s, the backlash that followed stopped most work in this area. The last couple of decades has seen both a revival of interest, and a more firmly grounded, experimentally justified use of information theory. We can view cognition as the process of transforming perceptions into information—where we use information in the colloquial sense of the word. This last clarification is one of the problems we run into when trying to use information theoretic principles to understand or analyze cognition. Information theory is mathematical, while cognition is a subjective phenomenon. It is relatively easy to discern a subjective connection between cognition and information; it is a different matter altogether to apply the rigor of information theory to the process of cognition. In this paper, we will look at the many ways in which people have tried to alleviate this problem. These approaches range from narrowing the focus to only quantifiable aspects of cognition or borrowing conceptual machinery from information theory to address issues of cognition. We describe applications of information theory across a range of cognition research, from neural coding to cognitive control and predictive coding.

Noise, Information and Fitness in Changing Environments

In this paper we discuss recent results regarding the question of adaptation to changing environments in intermediate timescales and the quantification of the amount of information a cell needs about its environment, connecting the theoretical approaches with relevant experimental results. We first show how advances in the study of noise in genetic circuits can inform a detailed description of intracellular information flow and allow for simplified descriptions of the phenotypic state of a cell. We then present the different types of strategies that cells can use to respond to changing environments, and what a quantitative description of this process implies about the long term fitness of the population. We present an early approach connecting the transmission of information to the average fitness, and then move on to a full model of the process. This model is then simplified to obtain analytical results for a few cases. We present the necessary notation but avoid technical detail as much as possible, as our goal is to emphasize the biological interpretation and significance of the mathematical results. We focus on how carefully constructed models can answer the long-standing objection to the use of information theory in biology based on decision-theoretic considerations of the difference between the amount of information and its fitness value.

Inform: Efficient Information-Theoretic Analysis of Collective Behaviors.

The study of collective behavior has traditionally relied on a variety of different methodological tools ranging from more theoretical methods such as population or game-theoretic models to empirical ones like Monte Carlo or multi-agent simulations. An approach that is increasingly being explored is the use of information theory as a methodological framework to study the flow of information and the statistical properties of collectives of interacting agents. While a few general purpose toolkits exist, most of the existing software for information theoretic analysis of collective systems is limited in scope. We introduce Inform, an open-source framework for efficient information theoretic analysis that exploits the computational power of a C library while simplifying its use through a variety of wrappers for common higher-level scripting languages. We focus on two such wrappers here: PyInform (Python) and rinform (R). Inform and its wrappers are cross-platform and general-purpose. They include classical information-theoretic measures, measures of information dynamics and information-based methods to study the statistical behavior of collective systems, and expose a lower-level API that allow users to construct measures of their own. We describe the architecture of the Inform framework, study its computational efficiency and use it to analyze three different case studies of collective behavior: biochemical information storage in regenerating planaria, nest-site selection in the ant Temnothorax rugatulus, and collective decision making in multi-agent simulations.

A brief introduction to mixed effects modelling and multi-model inference in ecology.

The use of linear mixed effects models (LMMs) is increasingly common in the analysis of biological data. Whilst LMMs offer a flexible approach to modelling a broad range of data types, ecological data are often complex and require complex model structures, and the fitting and interpretation of such models is not always straightforward. The ability to achieve robust biological inference requires that practitioners know how and when to apply these tools. Here, we provide a general overview of current methods for the application of LMMs to biological data, and highlight the typical pitfalls that can be encountered in the statistical modelling process. We tackle several issues regarding methods of model selection, with particular reference to the use of information theory and multi-model inference in ecology. We offer practical solutions and direct the reader to key references that provide further technical detail for those seeking a deeper understanding. This overview should serve as a widely accessible code of best practice for applying LMMs to complex biological problems and model structures, and in doing so improve the robustness of conclusions drawn from studies investigating ecological and evolutionary questions.

Identification of genome regions determining semen quality in Holstein-Friesian bulls using information theory

Use of information theory can be an alternative statistical approach to detect genome regions and candidate genes that are associated with livestock traits. The aim of this study was to verify the validity of the SNPs effects on some semen quality variables of bulls using entropy analysis. Records from 288 Holstein-Friesian bulls from one AI station were included. The following semen quality variables were analyzed: CASA kinematic variables of sperm (total motility, average path velocity, straight line velocity, curvilinear velocity, amplitude of lateral head displacement, beat cross frequency, straightness, linearity), sperm membrane integrity (plazmolema, mitochondrial function), sperm ATP content. Molecular data included 48,192 SNPs. After filtering (call rate = 0.95 and MAF = 0.05), 34,794 SNPs were included in the entropy analysis. The entropy and conditional entropy were estimated for each SNP. Conditional entropy quantifies the remaining uncertainty about values of the variable with the knowledge of SNP. The most informative SNPs for each variable were determined. The computations were performed using the R statistical package. A majority of the loci had relatively small contributions. The most informative SNPs for all variables were mainly located on chromosomes: 3, 4, 5 and 16. The results from the study indicate that important genome regions and candidate genes that determine semen quality variables in bulls are located on a number of chromosomes. Some detected clusters of SNPs were located in RNA (U6 and 5S_rRNA) for all the variables for which analysis occurred. Associations between PARK2 as well GALNT13 genes and some semen characteristics were also detected.

Strategic Information Processing from Behavioural Data in Iterated Games

Iterated games are an important framework of economic theory and application, at least since the original work of Axelrod’s computational tournaments of the early 80’s. Recent theoretical results have shown that games (the economic context) and game theory (the decision-making process) are both formally equivalent to computational logic gates. Here these results are extended to behavioural data obtained from an experiment in which rhesus monkeys sequentially played thousands of the “matching pennies” game, an empirical example similar to Axelrod’s tournaments in which algorithms played against one another. The results show that the monkeys exhibit a rich variety of behaviours, both between and within subjects when playing opponents of varying complexity. Despite earlier suggestions, there is no clear evidence that the win-stay, lose-switch strategy is used, however there is evidence of non-linear strategy-based interactions between the predictors of future choices. It is also shown that there is consistent evidence across protocols and across individuals that the monkeys extract non-markovian information, i.e., information from more than just the most recent state of the game. This work shows that the use of information theory in game theory can test important hypotheses that would otherwise be more difficult to extract using traditional statistical methods.

P270 Intra-locus gene structure in the genes of the HLA system

Aim This is a study of the genetic polymorphism of the different DNA segments of the HLA genes and how this genetic polymorphism affects genetic diversity. Methods Information Theory was used in the study of the intra-locus gene structure of HLA-A just as it had previously been used in the study of inter-loci genetic structures (see Tissue Antigens 2012 80:341). Information measures were used to evaluate the degree of genetic polymorphism of different gene segments as well as the relationships among them. The following genes are included in this study: A, B, C, DRB1, DRB345, DQB1, DQA1, DPB1 and DPA1. Results Entropy tables are given for all the gene segments from the 5 ′ UTR to the 3 ′ UTR for genes HLA-A, B, C, DQB1 and DPA1. For DRB1 and DRB345 only exon 1, not intron, and exon 2 through the 3 ′ UTR are included. For DQA1 segments from 5 ′ UTR through exon 4 are included. And for DPB1 exon 2 through exon 4 are included. 2 × 2 contingency tables showing the association between different gene segments with statistics to measure such association such as the chi-square a p-value, are also provided. In addition ‘well-conserved regions’ with high internal entropy are also identified and presented. Conclusions The proper characterisation of gene structure is the foundation to study the function of proteins encoded by the gene or genes in question. The use of Information Theory has proven to be a sound and robust method to study and measure genetic polymorphism and genetic diversity. Here we show how these tools can be used to evaluate HLA function, particularly in regard to epitope definition and peptide binding.

The use of information theory for the evaluation of biomarkers of aging and physiological age

The present work explores the application of information theoretical measures, such as entropy and normalized mutual information, for research of biomarkers of aging. The use of information theory affords unique methodological advantages for the study of aging processes, as it allows evaluating non-linear relations between biological parameters, providing the precise quantitative strength of those relations, both for individual and multiple parameters, showing cumulative or synergistic effect. Here we illustrate those capabilities utilizing a dataset on heart disease, including diagnostic parameters routinely available to physicians. The use of information-theoretical methods, utilizing normalized mutual information, revealed the exact amount of information that various diagnostic parameters or their combinations contained about the persons’ age. Based on those exact informative values for the correlation of measured parameters with age, we constructed a diagnostic rule (a decision tree) to evaluate physiological age, as compared to chronological age. The present data illustrated that younger subjects suffering from heart disease showed characteristics of people of higher age (higher physiological age). Utilizing information-theoretical measures, with additional data, it may be possible to create further clinically applicable information-theory-based markers and models for the evaluation of physiological age, its relation to age-related diseases and its potential modifications by therapeutic interventions.

Uncertainty quantification of environmental performance metrics in heterogeneous aquifers with long-range correlations

We investigate how the uncertainty stemming from disordered porous media that display long-range correlation in the hydraulic conductivity (K) field propagates to predictions of environmental performance metrics (EPMs). In this study, the EPMs are quantities that are of relevance to risk analysis and remediation, such as peak flux-averaged concentration, early and late arrival times among others. By using stochastic simulations, we quantify the uncertainty associated with the EPMs for a given disordered spatial structure of the K-field and identify the probability distribution function (PDF) model that best captures the statistics of the EPMs of interest. Results indicate that the probabilistic distribution of the EPMs considered in this study follows lognormal PDF. Finally, through the use of information theory, we reveal how the persistent/anti-persistent correlation structure of the K-field influences the EPMs and corresponding uncertainties.