Measures Of Information Transfer Research Articles

Application of advanced causal analyses to identify processes governing secondary organic aerosols

Understanding how different physical and chemical atmospheric processes affect the formation of fine particles has been a persistent challenge. Inferring causal relations between the various measured features affecting the formation of secondary organic aerosol (SOA) particles is complicated since correlations between variables do not necessarily imply causality. Here, we apply a state-of-the-art information transfer measure coupled with the Koopman operator framework to infer causal relations between isoprene epoxydiol SOA (IEPOX-SOA) and different chemistry and meteorological variables derived from detailed regional model predictions over the Amazon rainforest. IEPOX-SOA represents one of the most complex SOA formation pathways and is formed by the interactions between natural biogenic isoprene emissions and anthropogenic emissions affecting sulfate, acidity and particle water. Since the regional model captures the known relations of IEPOX-SOA with different chemistry and meteorological features, their simulated time series implicitly include their causal relations. We show that our causal model successfully infers the known major causal relations between total particle phase 2-methyl tetrols (the dominant component of IEPOX-SOA over the Amazon) and input features. We provide the first proof of concept that the application of our causal model better identifies causal relations compared to correlation and random forest analyses performed over the same dataset. Our work has tremendous implications, as our methodology of causal discovery could be used to identify unknown processes and features affecting fine particles and atmospheric chemistry in the Earth’s atmosphere.

Prediction of allosteric communication pathways in proteins.

Allostery in proteins is an essential phenomenon in biological processes. In this article, we present a computational model to predict paths of maximum information transfer between active and allosteric sites. In this information theoretic study, we use mutual information as the measure of information transfer, where transition probability of information from one residue to its contacting neighbors is proportional to the magnitude of mutual information between the two residues. Starting from a given residue and using a Hidden Markov Model, we successively determine the neighboring residues that eventually lead to a path of optimum information transfer. The Gaussian approximation of mutual information between residue pairs is adopted. The limits of validity of this approximation are discussed in terms of a nonlinear theory of mutual information and its reduction to the Gaussian form. Predictions of the model are tested on six widely studied cases, CheY Bacterial Chemotaxis, B-cell Lymphoma extra-large (Bcl-xL), Human proline isomerase cyclophilin A (CypA), Dihydrofolate reductase (DHFR), HRas GTPase and Caspase-1. The communication transmission rendering the propagation of local fluctuations from the active sites throughout the structure in multiple paths correlate well with the known experimental data. Distinct paths originating from the active site may likely represent a multi functionality such as involving more than one allosteric site and/or pre-existence of some other functional states. Our model is computationally fast and simple and can give allosteric communication pathways, which are crucial for the understanding and control of protein functionality. Supplementary data are available at Bioinformatics online.

A framework based on statistical analysis and stakeholders’ preferences to inform weighting in composite indicators

Composite Indicators (CIs, a.k.a. indices) are increasingly used as they can simplify interpretation of results by condensing the information of a plurality of underlying indicators in a single measure. This paper demonstrates that the strength of the correlations between the indicators is directly linked with their capacity to transfer information to the CI. A measure of information transfer from each indicator is proposed along with two weight-optimization methods, which allow the weights to be adjusted to achieve either a targeted or maximized information transfer. The tools presented in this paper are applied to a case study for resilience assessment of energy systems, demonstrating how they can support the tailored development of CIs. These findings enable analysts bridging the statistical properties of the index with the weighting preferences from the stakeholders. They can thus choose a weighting scheme and possibly modify the index while achieving a more consistent (by correlation) index.

On the α-q-Mutual Information and the α-q-Capacities.

The measures of information transfer which correspond to non-additive entropies have intensively been studied in previous decades. The majority of the work includes the ones belonging to the Sharma–Mittal entropy class, such as the Rényi, the Tsallis, the Landsberg–Vedral and the Gaussian entropies. All of the considerations follow the same approach, mimicking some of the various and mutually equivalent definitions of Shannon information measures, and the information transfer is quantified by an appropriately defined measure of mutual information, while the maximal information transfer is considered as a generalized channel capacity. However, all of the previous approaches fail to satisfy at least one of the ineluctable properties which a measure of (maximal) information transfer should satisfy, leading to counterintuitive conclusions and predicting nonphysical behavior even in the case of very simple communication channels. This paper fills the gap by proposing two parameter measures named the -q-mutual information and the -q-capacity. In addition to standard Shannon approaches, special cases of these measures include the -mutual information and the -capacity, which are well established in the information theory literature as measures of additive Rényi information transfer, while the cases of the Tsallis, the Landsberg–Vedral and the Gaussian entropies can also be accessed by special choices of the parameters and q. It is shown that, unlike the previous definition, the -q-mutual information and the -q-capacity satisfy the set of properties, which are stated as axioms, by which they reduce to zero in the case of totally destructive channels and to the (maximal) input Sharma–Mittal entropy in the case of perfect transmission, which is consistent with the maximum likelihood detection error. In addition, they are non-negative and less than or equal to the input and the output Sharma–Mittal entropies, in general. Thus, unlike the previous approaches, the proposed (maximal) information transfer measures do not manifest nonphysical behaviors such as sub-capacitance or super-capacitance, which could qualify them as appropriate measures of the Sharma–Mittal information transfer.

Effective connectivity in brain networks estimated using EEG signals is altered in children with ADHD

This study presents a methodology developed for estimating effective connectivity in brain networks (BNs) using multichannel scalp EEG recordings. The methodology uses transfer entropy as an information transfer measure to detect pair-wise directed information transfer between EEG signals within δ, θ, α, β and γ-bands. The developed methodology is then used to study the properties of directed BNs in children with attention-deficit hyperactivity disorder (ADHD) and compare them with that of the healthy controls using both statistical and receiver operating characteristic (ROC) analyses. The results indicate that directed information transfer between scalp EEG electrodes in the ADHD subjects differs significantly compared to the healthy ones. The results of the statistical and ROC analyses of frequency-specific graph measures demonstrate their highly discriminative ability between the two groups. Specifically, the graph measures extracted from the estimated directed BNs in the β-band show the highest discrimination between the ADHD and control groups. These findings are in line with the fact that β-band reflects active concentration, motor activity, and anxious mental states. The reported results show that the developed methodology has the capacity to be used for investigating patterns of directed BNs in neuropsychiatric disorders.

The effect of predation risk on group behaviour and information flow during repeated collective decisions

Making fast and accurate group decisions under uncertain and risky conditions is a fundamental problem for groups. Currently, there is little empirical evidence of how natural selection (such as environmental predation risk) has shaped the mechanisms of group decision making. We repeatedly tested individually marked guppies, Poecilia reticulata, from the upper and lower reaches of the Aripo and Turure rivers in a Y-maze and found that populations that had evolved under different predation regimes differed in their decision-making speed. All groups decreased decision time over successive trial rounds, but fish from low-predation environments did so at a greater rate than fish from high-predation environments. This effect was only significant when fish were tested in groups of eight, not when tested individually. Decision-making accuracy was not significantly different between high- and low-predation populations. Group behaviour differed according to predation risk and trial round, most notably with low-predation groups reducing the amount of positional switching over successive trial rounds at a greater rate than high-predation groups. While fish from both predator risk environments had repeatable within-shoal positioning behaviour as they swam down the maze, only fish from the high-predation groups had repeatable finishing positions. Meaningful measures of information transfer (mean pairwise transfer entropy) were only recorded in low-predation groups, and here there was a significant effect of source river on information flow within groups. Aripo groups increased information flow over successive trial rounds, while Turure groups did not. Variation in mean pairwise transfer entropy increased over successive trial rounds for both low-predation populations, suggesting that information used to make movement decisions became less homogeneously distributed and increasingly directed through a subset of individuals. Predation pressure is a ubiquitous selection force, and these findings may apply to a variety of natural social groups.

Measuring spectrally-resolved information transfer.

Information transfer, measured by transfer entropy, is a key component of distributed computation. It is therefore important to understand the pattern of information transfer in order to unravel the distributed computational algorithms of a system. Since in many natural systems distributed computation is thought to rely on rhythmic processes a frequency resolved measure of information transfer is highly desirable. Here, we present a novel algorithm, and its efficient implementation, to identify separately frequencies sending and receiving information in a network. Our approach relies on the invertible maximum overlap discrete wavelet transform (MODWT) for the creation of surrogate data in the computation of transfer entropy and entirely avoids filtering of the original signals. The approach thereby avoids well-known problems due to phase shifts or the ineffectiveness of filtering in the information theoretic setting. We also show that measuring frequency-resolved information transfer is a partial information decomposition problem that cannot be fully resolved to date and discuss the implications of this issue. Last, we evaluate the performance of our algorithm on simulated data and apply it to human magnetoencephalography (MEG) recordings and to local field potential recordings in the ferret. In human MEG we demonstrate top-down information flow in temporal cortex from very high frequencies (above 100Hz) to both similarly high frequencies and to frequencies around 20Hz, i.e. a complex spectral configuration of cortical information transmission that has not been described before. In the ferret we show that the prefrontal cortex sends information at low frequencies (4-8 Hz) to early visual cortex (V1), while V1 receives the information at high frequencies (> 125 Hz).

A Framework to Assess the Information Dynamics of Source EEG Activity and Its Application to Epileptic Brain Networks.

This study introduces a framework for the information-theoretic analysis of brain functional connectivity performed at the level of electroencephalogram (EEG) sources. The framework combines the use of common spatial patterns to select the EEG components which maximize the variance between two experimental conditions, simultaneous implementation of vector autoregressive modeling (VAR) with independent component analysis to describe the joint source dynamics and their projection to the scalp, and computation of information dynamics measures (information storage, information transfer, statistically significant network links) from the source VAR parameters. The proposed framework was tested on simulated EEGs obtained mixing source signals generated under different coupling conditions, showing its ability to retrieve source information dynamics from the scalp signals. Then, it was applied to investigate scalp and source brain connectivity in a group of children manifesting episodes of focal and generalized epilepsy; the analysis was performed on EEG signals lasting 5 s, collected in two consecutive windows preceding and one window following each ictal episode. Our results show that generalized seizures are associated with a significant decrease from pre-ictal to post-ictal periods of the information stored in the signals and of the information transferred among them, reflecting reduced self-predictability and causal connectivity at the level of both scalp and source brain dynamics. On the contrary, in the case of focal seizures the scalp EEG activity was not discriminated across conditions by any information measure, while source analysis revealed a tendency of the measures of information transfer to increase just before seizures and to decrease just after seizures. These results suggest that focal epileptic seizures are associated with a reorganization of the topology of EEG brain networks which is only visible analyzing connectivity among the brain sources. Our findings emphasize the importance of EEG modeling approaches able to deal with the adverse effects of volume conduction on brain connectivity analysis, and their potential relevance to the development of strategies for prediction and clinical treatment of epilepsy.

An information theoretic approach to insulin sensing by human kidney podocytes

Podocytes are key components of the glomerular filtration barrier (GFB). They are insulin-responsive but can become insulin-resistant, causing features of the leading global cause of kidney failure, diabetic nephropathy. Insulin acts via insulin receptors to control activities fundamental to GFB integrity, but the amount of information transferred is unknown. Here we measure this in human podocytes, using information theory-derived statistics that take into account cell-cell variability. High content imaging was used to measure insulin effects on Akt, FOXO and ERK. Mutual Information (MI) and Channel Capacity (CC) were calculated as measures of information transfer. We find that insulin acts via noisy communication channels with more information flow to Akt than to ERK. Information flow estimates were increased by consideration of joint sensing (ERK and Akt) and response trajectory (live cell imaging of FOXO1-clover translocation). Nevertheless, MI values were always <1Bit as most information was lost through signaling. Constitutive PI3K activity is a predominant feature of the system that restricts the proportion of CC engaged by insulin. Negative feedback from Akt supressed this activity and thereby improved insulin sensing, whereas sensing was robust to manipulation of feedforward signaling by inhibiting PI3K, PTEN or PTP1B. The decisions made by individual podocytes dictate GFB integrity, so we suggest that understanding the information on which the decisions are based will improve understanding of diabetic kidney disease and its treatment.

Information Transfer in Linear Multivariate Processes Assessed through Penalized Regression Techniques: Validation and Application to Physiological Networks.

The framework of information dynamics allows the dissection of the information processed in a network of multiple interacting dynamical systems into meaningful elements of computation that quantify the information generated in a target system, stored in it, transferred to it from one or more source systems, and modified in a synergistic or redundant way. The concepts of information transfer and modification have been recently formulated in the context of linear parametric modeling of vector stochastic processes, linking them to the notion of Granger causality and providing efficient tools for their computation based on the state–space (SS) representation of vector autoregressive (VAR) models. Despite their high computational reliability these tools still suffer from estimation problems which emerge, in the case of low ratio between data points available and the number of time series, when VAR identification is performed via the standard ordinary least squares (OLS). In this work we propose to replace the OLS with penalized regression performed through the Least Absolute Shrinkage and Selection Operator (LASSO), prior to computation of the measures of information transfer and information modification. First, simulating networks of several coupled Gaussian systems with complex interactions, we show that the LASSO regression allows, also in conditions of data paucity, to accurately reconstruct both the underlying network topology and the expected patterns of information transfer. Then we apply the proposed VAR-SS-LASSO approach to a challenging application context, i.e., the study of the physiological network of brain and peripheral interactions probed in humans under different conditions of rest and mental stress. Our results, which document the possibility to extract physiologically plausible patterns of interaction between the cardiovascular, respiratory and brain wave amplitudes, open the way to the use of our new analysis tools to explore the emerging field of Network Physiology in several practical applications.

On Data-Driven Computation of Information Transfer for Causal Inference in Discrete-Time Dynamical Systems

In this paper, we provide a novel approach to capture causal interaction in a dynamical system from time series data. In Sinha and Vaidya (in: IEEE conference on decision and control, pp 7329–7334, 2016), we have shown that the existing measures of information transfer, namely directed information, Granger causality and transfer entropy, fail to capture the causal interaction in a dynamical system and proposed a new definition of information transfer that captures direct causal interactions. The novelty of the information transfer definition used in this paper is the fact that it can differentiate between direct and indirect influences Sinha and Vaidya (2016). The main contribution of this paper is to show that the proposed definition of information transfers in Sinha and Vaidya (2016) and Sinha and Vaidya (in: Indian control conference, pp 303–308, 2017) can be computed from time series data, and thus, the direct influences in a dynamical system can be identified from time series data. We use transfer operator theoretic framework, involving Perron–Frobenius and Koopman operators for the data-driven approximation of the system dynamics and computation of information transfer. Several examples, involving linear and nonlinear system dynamics, are presented to verify the efficiency of the developed algorithm.

Learning in Feedforward Neural Networks Accelerated by Transfer Entropy.

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance.

A study for multiscale information transfer measures based on conditional mutual information.

As the big data science develops, efficient methods are demanded for various data analysis. Granger causality provides the prime model for quantifying causal interactions. However, this theoretic model does not meet the requirement for real-world data analysis, because real-world time series are diverse whose models are usually unknown. Therefore, model-free measures such as information transfer measures are strongly desired. Here, we propose the multi-scale extension of conditional mutual information measures using MORLET wavelet, which are named the WM and WPM. The proposed measures are computational efficient and interpret information transfer by multi-scales. We use both synthetic data and real-world examples to demonstrate the efficiency of the new methods. The results of the new methods are robust and reliable. Via the simulation studies, we found the new methods outperform the wavelet extension of transfer entropy (WTE) in both computational efficiency and accuracy. The features and properties of the proposed measures are also discussed.

A local approach for information transfer

In this work, a strategy to estimate the information transfer between the elements of a complex system, from the time series associated to the evolution of this elements, is presented. By using the nearest neighbors of each state, the local approaches of the deterministic dynamical rule generating the data and the probability density functions, both marginals and conditionals, necessaries to calculate some measures of information transfer, are estimated.The performance of the method using numerically simulated data and real signals is exposed.

Improving Handoffs Between Operating Room and Pediatric Intensive Care Teams: Before and After Study.

Introduction:Patient transfer between teams and units is known to be a high-risk event for miscommunication and therefore error. We instituted a quality improvement initiative to formalize patient handoffs from the operating room (OR) to the Pediatric Intensive Care Unit (PICU). We hypothesized that measures of information transfer would improve.Methods:In this before and after study, a multidisciplinary team developed a standardized handoff protocol (including a checklist) instituted in the Dartmouth PICU over the summer of 2016. We directly observed pediatric admissions from OR to PICU and collected data on information transfer and patient outcome metrics both before and after the institution of the handoff protocol at the time of transfer (intervention).Results:We directly observed 52 handoffs (29 preintervention, 23 postintervention). The mean patient age was 9.3 years (SD, 6.5), with 55% male. Preintervention the average information transfer was 56% (upper control limit, 76%; lower control limit, 36%), whereas postintervention it was 81% (upper control limit, 97%, lower control limit, 65%). The improvement in information transfer postintervention was statistically significant (P < 0.001). There was no statistically significant change in maximum pain score in the first 6 hours after admission (preintervention, 4.5, SD 3.9; postintervention, 2.9, SD 1.3, P = 0.15). There was no difference in the time required for handoff pre- versus postintervention (8.7 minutes, SD 5.5 versus 10.1 minutes, SD 4.6, P = 0.34).Conclusion:Standardization of OR to PICU patient transfers using a predetermined checklist at the time of handoff can improve the completeness of information transfer without increasing the length of the handoff.

An upper bound on the rate of information transfer in optical vortex beams

Light endowed with orbital angular momentum, commonly termed optical vortex light, has an azimuthal phase indexed by the orbital quantum number l. In contrast to the two basis states of the optical spin angular momentum, the interest in the information content of optical vortex beams is centred on the assumption that forms a countably infinite set of basis states. The recent experimental observation that group velocity is inversely proportional to l provides a theoretical basis for a practical measure of information transfer. This letter sets an upper bound on that measure in free-space propagation.

Recognizing Information Feature Variation: Message Importance Transfer Measure and Its Applications in Big Data.

Information transfer that characterizes the information feature variation can have a crucial impact on big data analytics and processing. Actually, the measure for information transfer can reflect the system change from the statistics by using the variable distributions, similar to Kullback-Leibler (KL) divergence and Renyi divergence. Furthermore, to some degree, small probability events may carry the most important part of the total message in an information transfer of big data. Therefore, it is significant to propose an information transfer measure with respect to the message importance from the viewpoint of small probability events. In this paper, we present the message importance transfer measure (MITM) and analyze its performance and applications in three aspects. First, we discuss the robustness of MITM by using it to measuring information distance. Then, we present a message importance transfer capacity by resorting to the MITM and give an upper bound for the information transfer process with disturbance. Finally, we apply the MITM to discuss the queue length selection, which is the fundamental problem of caching operation on mobile edge computing.

Directionality indices: Testing information transfer with surrogate correction.

Directionality indices can be used as an indicator of the asymmetry in coupling between systems and have found particular application in relation to neurological systems. The directionality index between two systems is a function of measures of information transfer in both directions. Here we illustrate that before inferring the directionality of coupling it is first necessary to consider the use of appropriate tests of significance. We propose a surrogate corrected directionality index which incorporates such testing. We also highlight the differences between testing the significance of the directionality index itself versus testing the individual measures of information transfer in each direction. To validate the approach we compared two different methods of estimating coupling, both of which have previously been used to estimate directionality indices. These were the modeling-based evolution map approach and a conditional mutual information (CMI) method for calculating dynamic information rates. For the CMI-based approach we also compared two different methods for estimating the CMI, an equiquantization-based estimator and a k-nearest neighbors estimator.

Pre-processing and transfer entropy measures in motor neurons controlling limb movements.

Directed information transfer measures are increasingly being employed in modeling neural system behavior due to their model-free approach, applicability to nonlinear and stochastic signals, and the potential to integrate repetitions of an experiment. Intracellular physiological recordings of graded synaptic potentials provide a number of additional challenges compared to spike signals due to non-stationary behaviour generated through extrinsic processes. We therefore propose a method to overcome this difficulty by using a preprocessing step based on Singular Spectrum Analysis (SSA) to remove nonlinear trends and discontinuities. We apply the method to intracellular recordings of synaptic responses of identified motor neurons evoked by stimulation of a proprioceptor that monitors limb position in leg of the desert locust. We then apply normalized delayed transfer entropy measures to neural responses evoked by displacements of the proprioceptor, the femoral chordotonal organ, that contains sensory neurones that monitor movements about the femoral-tibial joint. We then determine the consistency of responses within an individual recording of an identified motor neuron in a single animal, between repetitions of the same experiment in an identified motor neurons in the same animal and in repetitions of the same experiment from the same identified motor neuron in different animals. We found that delayed transfer entropy measures were consistent for a given identified neuron within and between animals and that they predict neural connectivity for the fast extensor tibiae motor neuron.

Instantaneous Transfer Entropy for the Study of Cardiovascular and Cardiorespiratory Nonstationary Dynamics.

Measures of transfer entropy (TE) quantify the direction and strength of coupling between two complex systems. Standard approaches assume stationarity of the observations, and therefore are unable to track time-varying changes in nonlinear information transfer with high temporal resolution. In this study, we aim to define and validate novel instantaneous measures of TE to provide an improved assessment of complex nonstationary cardiorespiratory interactions. We here propose a novel instantaneous point-process TE (ipTE) and validate its assessment as applied to cardiovascular and cardiorespiratory dynamics. In particular, heartbeat and respiratory dynamics are characterized through discrete time series, and modeled with probability density functions predicting the time of the next physiological event as a function of the past history. Likewise, nonstationary interactions between heartbeat and blood pressure dynamics are characterized as well. Furthermore, we propose a new measure of information transfer, the instantaneous point-process information transfer (ipInfTr), which is directly derived from point-process-based definitions of the Kolmogorov-Smirnov distance. Analysis on synthetic data, as well as on experimental data gathered from healthy subjects undergoing postural changes confirms that ipTE, as well as ipInfTr measures are able to dynamically track changes in physiological systems coupling. This novel approach opens new avenues in the study of hidden, transient, nonstationary physiological states involving multivariate autonomic dynamics in cardiovascular health and disease. The proposed method can also be tailored for the study of complex multisystem physiology (e.g., brain-heart or, more in general, brain-body interactions).