Causation Entropy Research Articles

Contrasting topologies of synchronous and asynchronous functional brain networks

ABSTRACT We generated asynchronous functional networks (aFNs) using a novel method called optimal causation entropy and compared aFN topology with the correlation-based synchronous functional networks (sFNs), which are commonly used in network neuroscience studies. Functional magnetic resonance imaging (fMRI) time series from 212 participants of the National Consortium on Alcohol and Neurodevelopment in Adolescence study were used to generate aFNs and sFNs. As a demonstration of how aFNs and sFNs can be used in tandem, we used multivariate mixed effects models to determine whether age interacted with node efficiency to influence connection probabilities in the two networks. After adjusting for differences in network density, aFNs had higher global efficiency but lower local efficiency than the sFNs. In the aFNs, nodes with the highest outgoing global efficiency tended to be in the brainstem and orbitofrontal cortex. aFN nodes with the highest incoming global efficiency tended to be members of the default mode network in sFNs. Age interacted with node global efficiency in aFNs and node local efficiency in sFNs to influence connection probability. We conclude that the sFN and aFN both offer information about functional brain connectivity, which the other type of network does not.

Probing Asymmetric Interactions with Time-Separated Mutual Information: A Case Study Using Golden Shiners.

Leader-follower modalities and other asymmetric interactions that drive the collective motion of organisms are often quantified using information theory metrics like transfer or causation entropy. These metrics are difficult to accurately evaluate without a much larger number of data than is typically available from a time series of animal trajectories collected in the field or from experiments. In this paper, we use a generalized leader-follower model to argue that the time-separated mutual information between two organism positions can serve as an alternative metric for capturing asymmetric correlations that is much less data intensive and more accurately estimated by popular k-nearest neighbor algorithms than transfer entropy. Our model predicts a local maximum of this mutual information at a time separation value corresponding to the fundamental reaction timescale of the follower organism. We confirm this prediction by analyzing time series trajectories recorded for a pair of golden shiner fish circling an annular tank.

Contrasting topologies of synchronous and asynchronous functional brain networks.

We generated asynchronous functional networks (aFNs) using a novel method called optimal causation entropy (oCSE) and compared aFN topology to the correlation-based synchronous functional networks (sFNs) which are commonly used in network neuroscience studies. Functional magnetic resonance imaging (fMRI) time series from 212 participants of the National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA) study were used to generate aFNs and sFNs. As a demonstration of how aFNs and sFNs can be used in tandem, we used multivariate mixed effects models to determine whether age interacted with node efficiency to influence connection probabilities in the two networks. After adjusting for differences in network density, aFNs had higher global efficiency but lower local efficiency than the sFNs. In the aFNs, nodes with the highest outgoing global efficiency tended to be in the brainstem and orbitofrontal cortex. aFN nodes with the highest incoming global efficiency tended to be members of the Default Mode Network (DMN) in sFNs. Age interacted with node global efficiency in aFNs and node local efficiency in sFNs to influence connection probability. We conclude that the sFN and aFN both offer information about functional brain connectivity which the other type of network does not.

Lag and duration of leader-follower relationships in mixed traffic using causal inference.

This study presents comprehensive analysis of car-following behavior on roads, utilizing Granger causality and transfer entropy techniques to enhance the validity of existing car-following models. It was found that most leader-follower relationships exhibit a delay in lateral movement by 4-5 s and last for short periods of around 3-5 s. These patterns are exhibited for all types of relationship found in the dataset, as well as for followers of all types. These findings imply that lateral movement reactions are governed by a different set of rules from braking and acceleration reactions, and the advantage in following lateral changes is short-lived. This also suggests that mixed traffic conditions may force drivers to slow down and calibrate reactions, as well as limiting the speed advantage gained by following a leader. Our methods were verified against random sampling as a method of selecting leader-follower pairs, decreasing the percent error in predicted speeds by 9.5% using the optimal velocity car-following model. The study concludes with a set of recommendations for future work, including the use of a diversity of car-following models for simulation and the use of causation entropy to distinguish between direct and indirect influences.

CEBoosting: Online sparse identification of dynamical systems with regime switching by causation entropy boosting.

Regime switching is ubiquitous in many complex dynamical systems with multiscale features, chaotic behavior, and extreme events. In this paper, a causation entropy boosting (CEBoosting) strategy is developed to facilitate the detection of regime switching and the discovery of the dynamics associated with the new regime via online model identification. The causation entropy, which can be efficiently calculated, provides a logic value of each candidate function in a pre-determined library. The reversal of one or a few such causation entropy indicators associated with the model calibrated for the current regime implies the detection of regime switching. Despite the short length of each batch formed by the sequential data, the accumulated value of causation entropy corresponding to a sequence of data batches leads to a robust indicator. With the detected rectification of the model structure, the subsequent parameter estimation becomes a quadratic optimization problem, which is solved using closed analytic formulas. Using the Lorenz 96 model, it is shown that the causation entropy indicator can be efficiently calculated, and the method applies to moderately large dimensional systems. The CEBoosting algorithm is also adaptive to the situation with partial observations. It is shown via a stochastic parameterized model that the CEBoosting strategy can be combined with data assimilation to identify regime switching triggered by the unobserved latent processes. In addition, the CEBoosting method is applied to a nonlinear paradigm model for topographic mean flow interaction, demonstrating the online detection of regime switching in the presence of strong intermittency and extreme events.

Data-driven statistical reduced-order modeling and quantification of polycrystal mechanics leading to porosity-based ductile damage

Predicting the process of porosity-based ductile damage in polycrystalline metallic materials is an essential practical topic. Ductile damage and its precursors are represented by extreme values in stress and material state quantities, the spatial probability density function (PDF) of which are highly non-Gaussian with strong fat tails. Traditional deterministic forecasts utilizing sophisticated continuum-based physical models generally lack in representing the statistics of structural evolution during material deformation. Computational tools which do represent complex structural evolution are typically expensive. The inevitable model error and the lack of uncertainty quantification may also induce significant forecast biases, especially in predicting the extreme events associated with ductile damage. In this paper, a data-driven statistical reduced-order modeling framework is developed to provide a probabilistic forecast of the deformation process of a polycrystal aggregate leading to porosity-based ductile damage with uncertainty quantification. The framework starts with computing the time evolution of the leading few moments of specific state variables from the spatiotemporal solution of full-field polycrystal simulations. Then a sparse model identification algorithm based on causation entropy, including essential physical constraints, is utilized to discover the governing equations of these moments. An approximate solution of the time evolution of the PDF is obtained from the predicted moments exploiting the maximum entropy principle. Numerical experiments based on polycrystal realizations of a representative body-centered cubic (BCC) tantalum illustrate a skillful reduced-order model in characterizing the time evolution of the non-Gaussian PDF of the von Mises stress and quantifying the probability of extreme events. The learning process also reveals that the mean stress is not simply an additive forcing to drive the higher-order moments and extreme events. Instead, it interacts with the latter in a strongly nonlinear and multiplicative fashion. In addition, the calibrated moment equations provide a reasonably accurate forecast when applied to the realizations outside the training data set, indicating the robustness of the model and the skill for extrapolation. Finally, an information-based measurement is employed to quantitatively justify that the leading four moments are sufficient to characterize the crucial highly non-Gaussian features throughout the entire deformation history considered.

Causation Entropy En Route to Re-Synchronization Unfolds the Structure of Chaotic Dynamical Networks

Revealing the interactions between the individual components in complex networks utilizing limited observations is a subject of scientific interest. Determination of connectivity of the network is important, specifically. When the network consists of dynamically evolving nodes that tend to synchronize, the problem is especially hard because the observations from synchronizing nodes are identical eventually and do not reveal much about the structure. Here, we present a method for reconstructing unidirectional chaotic synchronized networks based on average integrated causation entropy. We show that if we inject random information via impulsive perturbations into the individual systems to destroy the synchronization briefly, that can lead to the prediction of the network structure en route to re-synchronization. We propose an algorithm based on the properties of average integrated causation entropy, a measure of information transferred between two nodes relative to a third node. The performance of the proposed algorithm has been demonstrated with chaotic networks to show that it can accurately reconstruct the structure of chaotic dynamical systems.

Data-driven learning of Boolean networks and functions by optimal causation entropy principle

Boolean functions, and networks thereof, are useful for analysis of complex data systems, including from biological systems, bioinformatics, decision making, medical fields, and finance. However, automated learning of a Boolean networked function, from data, is a challenging task due in part to the large number of unknown structures of the network and the underlying functions. In this paper, we develop a new information theoretic methodology, called Boolean optimal causation entropy, that we show is significantly more efficient than previous approaches. Our method is computationally efficient and also resilient to noise. Furthermore, it allows for selection of features that best explains the process, described as a networked Boolean function reduced-order model. We highlight our method to the feature selection in several real-world examples: (1) diagnosis of urinary diseases, (2) cardiac single proton emission computed tomography diagnosis, (3) informative positions in the game Tic-Tac-Toe, and (4) risk causality analysis of loans in default status.

System Identification of an Actuated Inclined Ball Mechanism Via Causation Entropy

Abstract Grey-box and black-box dynamic models are routinely used to model the behavior of real-world dynamic systems. When creating such models, the identification of an accurate model structure (often referred to as covariate selection, feature selection, or sparsity identification) is a critical step required to achieve suitable predictive performance by minimizing the effects of overfitting. Recently, causation entropy has been shown to be quite useful in data-driven covariate selection as it provides a mechanism to measure the causal relationships between the set of covariates and the state dynamics. This work extends previous results by applying the causation entropy covariate selection technique to data from an experimental nonlinear system consisting of a ball rolling on an actuated inclined ramp. Data collected from the system is processed by the causation entropy-based algorithm and covariate selection is performed on a black-box dynamic model. The resulting optimized model is shown to provide better predictive performance than an optimized black-box model which includes extraneous covariates. This study represents the first application of causation entropy-based covariate selection to real-world experimental data, illustrating its use as a practical system identification method.

Interaction networks from discrete event data by Poisson multivariate mutual information estimation and information flow with applications from gene expression data

In this work, we introduce a new methodology for inferring the interaction structure of discrete valued time series which are Poisson distributed. While most related methods are premised on continuous state stochastic processes, in fact, discrete and counting event oriented stochastic process are natural and common, so called time-point processes. An important application that we focus on here is gene expression, where it is often assumed that the data is generated from a multivariate Poisson distribution. Nonparameteric methods such as the popular k-nearest neighbors are slow converging for discrete processes, and thus data hungry. Now, with the new multi-variate Poisson estimator developed here as the core computational engine, the causation entropy (CSE) principle, together with the associated greedy search algorithm optimal CSE (oCSE) allows us to efficiently infer the true network structure for this class of stochastic processes that were previously not practical. We illustrate the power of our method, first in benchmarking with synthetic datum, and then by inferring the genetic factors network from a breast cancer micro-ribonucleic acid sequence count data set. We show the Poisson oCSE gives the best performance among the tested methods and discovers previously known interactions on the breast cancer data set.

Abnormality monitoring and causality analysis based on KF-PDC and IACE in blast furnace ironmaking process

ABSTRACT Blast furnace (BF) ironmaking is a highly complicated process with multi-variable nonlinear coupling and multi-mode characteristics. In this article, a developed kernel function partial derivative contribution (KF-PDC) is proposed for abnormality location, which makes up for the deficiency of linear multivariate statistical process monitoring (MSPM) and de-redundancies the variable candidate set of causality analysis. Then, to eliminate the influence of multi-mode, an online interval adaptive causation entropy (IACE) is established to analyse the cause–effect relationships of candidate abnormal variables, which contributes to distinguishing the direct and indirect causality, and the Haar wavelet based on the sliding window (HWSW) is constructed for the segmentation of different modes online. Finally, a case study using actual industrial BF ironmaking data illustrates that the monitoring method can better capture the abnormal furnace conditions and effectively obtain the root cause and propagation path.

Entropic regression with neurologically motivated applications

The ultimate goal of cognitive neuroscience is to understand the mechanistic neural processes underlying the functional organization of the brain. The key to this study is understanding the structure of both the structural and functional connectivity between anatomical regions. In this paper, we use an information theoretic approach, which defines direct information flow in terms of causation entropy, to improve upon the accuracy of the recovery of the true network structure over popularly used methods for this task such as correlation and least absolute shrinkage and selection operator regression. The method outlined above is tested on synthetic data, which is produced by following previous work in which a simple dynamical model of the brain is used, simulated on top of a real network of anatomical brain regions reconstructed from diffusion tensor imaging. We demonstrate the effectiveness of the method of AlMomani et al. [Chaos 30, 013107 (2020)] when applied to data simulated on the realistic diffusion tensor imaging network, as well as on randomly generated small-world and Erdös–Rényi networks.

A novel prediction model of multi-layer symbolic pattern network: Based on causation entropy

Faced with the high-dimensional characteristics of time series, symbolization is usually used to reduce the dimensionality and effectively eliminate redundancy and irrelevant features. Several existing network prediction methods, which based on the topology of complex networks, only consider the target sequence itself, which ignored the influence of related factors. In this paper, we improve the single-layer network and construct a Multi-layer Symbolic Pattern Network (MSPN) prediction model by considering the interaction between different variables with time lag. The basic idea is to link the directed weighted networks based on causation entropy (CSE), then extract the multi-layer network topology to jointly predict the target sequence. The 2018/1–2018/12 fluctuation trend of crude oil futures prices are predicted, with auxiliary variables including crude oil supply, demand and the three major U.S. stock indexes (S&P 500 Index, NASDAQ Composite Index, and Dow Jones Industrial Average). The results show that the multi-layer network improves the directional prediction accuracy of oil price over the single-layer network, where demand and Dow Jones Industrial Average’s auxiliary perform best. It is worth noting that considering auxiliary variables can be effective in improving the accuracy of the model. The more auxiliary layers are not the better, generally considering two auxiliary layers, that is, the three-layer network prediction model can achieve higher accuracy.

Practical Considerations for Use of Causation Entropy in Sparsity Identification

Abstract The selection of model structure is an important step in system identification for nonlinear systems in cases where the model form is not known a priori. This process, sometimes called covariate selection or sparsity identification, involves the selection of terms in the dynamic model and is performed prior to parameter estimation. Previous work has shown the applicability of an information theory quantity known as causation entropy in performing sparsity identification. While prior work established the overall feasibility of using causation entropy to eliminate extraneous terms in a model, key questions remained regarding practical implementation. This paper builds on previous work to explore key practical considerations of causation entropy sparsity identification. First, the effect of data size is explored through both analysis and simulation, and general guidance is provided on how much data is necessary to produce accurate causation entropy estimates. Second, the effects of measurement noise and model discretization error are investigated, showing that both cause degradation of the causation entropy estimation accuracy but in opposite ways. These practical effects and trends are illustrated on several example nonlinear systems. Overall, results show that the causation entropy approach is a practical technique for sparsity identification particularly in light of the guidelines presented here for data size selection and handling of error sources.

Quantifying interactions among car drivers using information theory.

Information-theoretic quantities have found wide applications in understanding interactions in complex systems primarily due to their non-parametric nature and ability to capture non-linear relationships. Increasingly popular among these tools is conditional transfer entropy, also known as causation entropy. In the present work, we leverage this tool to study the interaction among car drivers for the first time. Specifically, we investigate whether a driver responds to its immediate front and its immediate rear car to the same extent and whether we can separately quantify these responses. Using empirical data, we learn about the important features related to human driving behavior. Results demonstrate the evidence that drivers respond to both front and rear cars, and the response to their immediate front car increases in the presence of jammed traffic. Our approach provides a data-driven perspective to study interactions and is expected to aid in analyzing traffic dynamics.

NIMCE: A Gene Regulatory Network Inference Approach Based on Multi Time Delays Causal Entropy.

Gene regulatory networks (GRNs)are involved in various biological processes, such as cell cycle, differentiation and apoptosis. The existing large amount of expression data, especially the time-series expression data, provide a chance to infer GRNs by computational methods. These data can reveal the dynamics of gene expression and imply the regulatory relationships among genes. However, identify the indirect regulatory links is still a big challenge as most studies treat time points as independent observations, while ignoring the influences of time delays. In this study, we propose a GRN inference method based on information-theory measure, called NIMCE. NIMCE incorporates the transfer entropy to measure the regulatory links between each pair of genes, then applies the causation entropy to filter indirect relationships. In addition, NIMCE applies multi time delays to identify indirect regulatory relationships from candidate genes. Experiments on simulated and colorectal cancer data show NIMCE outperforms than other competing methods. All data and codes used in this study are publicly available at https://github.com/CSUBioGroup/NIMCE.

Can we detect clusters of chaotic dynamical networks via causation entropy?

It is known that chaotic dynamical systems in the coupled networks can synchronize, and they can even form clusters. Our study addresses the issue of determining the membership information of continuous-time dynamical networks forming clusters. We observe the output vectors of individual systems in the networks and reconstruct the state space according to Takens' embedding theorem. Afterward, we estimate the information-theoretic measures in the reconstructed state space. We propose the average integrated causation entropy as a model-free distinctive measure to distinguish the clusters in the network using the k-means clustering algorithm. We have demonstrated the proposed procedure on three networks that contain Chua systems. The results indicate that we can determine the members of clusters and the membership information from the data, conclusively.

Probabilistic causal inference for coordinated movement of pigeon flocks

One of the most striking examples of natural animal behavior is the highly ordered coordinated group movement, where inter-agent interaction is considered as a key factor tuning the coordination. To understand the interaction mechanism, a long-standing challenge is to reveal the causal relationship among the group individuals. In this study, we propose a causal inference method from the viewpoint of information theory. More precisely, we extend conditional information entropy of pairwise individuals with time delay, and subsequently induce causation entropy which quantifies the causal dependence of two individuals subject to a condition set consisting of other neighbors. Moreover, we analyze the high-resolution GPS data of three pigeon flocks to construct the interaction networks accordingly. We dynamically analyze the interaction characteristics and interestingly observe that the individuals closer to the mass center and the average velocity direction are more influential to the others. This study may shed some light onto the research of collective behaviors.

Decoding collective communications using information theory tools.

Organisms have evolved sensory mechanisms to extract pertinent information from their environment, enabling them to assess their situation and act accordingly. For social organisms travelling in groups, like the fish in a school or the birds in a flock, sharing information can further improve their situational awareness and reaction times. Data on the benefits and costs of social coordination, however, have largely allowed our understanding of why collective behaviours have evolved to outpace our mechanistic knowledge of how they arise. Recent studies have begun to correct this imbalance through fine-scale analyses of group movement data. One approach that has received renewed attention is the use of information theoretic (IT) tools like mutual information, transfer entropy and causation entropy, which can help identify causal interactions in the type of complex, dynamical patterns often on display when organisms act collectively. Yet, there is a communications gap between studies focused on the ecological constraints and solutions of collective action with those demonstrating the promise of IT tools in this arena. We attempt to bridge this divide through a series of ecologically motivated examples designed to illustrate the benefits and challenges of using IT tools to extract deeper insights into the interaction patterns governing group-level dynamics. We summarize some of the approaches taken thus far to circumvent existing challenges in this area and we conclude with an optimistic, yet cautionary perspective.

Inference and analysis of gene regulatory networks using data mining techniques

Causality draws the relationship between the cause and the effect among a pair of items. Causality inference in Gene Regulatory Networks (GRNs) is a challenging task due to the limits of appropriate data and large dataset size of real networks. Such an inference is very important because it might give the blueprint of how a complex organism operates which might further reveal the probable cause of some of the terminal diseases. A number of techniques have been proposed and applied to infer causal relationships in various domains. However, they are not specific to regulatory network inference. In our work, we made an empirical study on a few selective methods in inferring causal GRNs. To assess the candidate methods in terms of their quality of inference we used time-series expression data from the DREAM challenges. Our findings revealed that among the selected causality finding techniques, Causation Entropy gives the best result, however, it is computationally expensive. We used the second-best approach, Transfer Entropy, to infer casual GRN of breast cancer datasets. The topological analysis of the network revealed that top out-degree genes such as SLC39A5 which are considered central genes, plays an important role in cancer progression. The main intention behind this research work was to develop an effective inference method for causal Gene Regulatory Network which is computationally less expensive and also considers the effects of time-delay during regulation of one gene by the other. We proposed a novel time delayed inferring method for GRNs using association rule mining. We tried to infer the regulatory sign of the edges i.e. whether influencing positively or negatively. Comparative results highlight that our proposed technique produced better inference results in comparison to several of the existing techniques. We applied the proposed method to perform post inference analysis on malignant cancer network and benign network. Various networks parameters like the degree distribution, average path length, clustering coefficient, network re-wiring structure analysis, network topological variation analysis, and connectivity variation analysis are performed to identify the differentially expressed genes between the two networks. To avoid inference biasness and to obtain a consensus based conclusion we made a comparative analysis between our proposed method and transfer entropy based method and also computed the average score. Such an analysis of the network has revealed four genes, namely, STON 2 , UMODL 1 , STK 11 , and COBL which are already under investigation for its role in various types of cancer as seen from various researches.