Local Transfer Entropy Research Articles

Information-theoretic analyses of neural data to minimize the effect of researchers' assumptions in predictive coding studies.

Studies investigating neural information processing often implicitly ask both, which processing strategy out of several alternatives is used and how this strategy is implemented in neural dynamics. A prime example are studies on predictive coding. These often ask whether confirmed predictions about inputs or prediction errors between internal predictions and inputs are passed on in a hierarchical neural system-while at the same time looking for the neural correlates of coding for errors and predictions. If we do not know exactly what a neural system predicts at any given moment, this results in a circular analysis-as has been criticized correctly. To circumvent such circular analysis, we propose to express information processing strategies (such as predictive coding) by local information-theoretic quantities, such that they can be estimated directly from neural data. We demonstrate our approach by investigating two opposing accounts of predictive coding-like processing strategies, where we quantify the building blocks of predictive coding, namely predictability of inputs and transfer of information, by local active information storage and local transfer entropy. We define testable hypotheses on the relationship of both quantities, allowing us to identify which of the assumed strategies was used. We demonstrate our approach on spiking data collected from the retinogeniculate synapse of the cat (N = 16). Applying our local information dynamics framework, we are able to show that the synapse codes for predictable rather than surprising input. To support our findings, we estimate quantities applied in the partial information decomposition framework, which allow to differentiate whether the transferred information is primarily bottom-up sensory input or information transferred conditionally on the current state of the synapse. Supporting our local information-theoretic results, we find that the synapse preferentially transfers bottom-up information.

Numerical investigation of entropy generation of turbulent flow in twisted tri-lobed tubes

Twisted tri-lobed tubes have garnered attention due to their exceptional heat transfer efficiency and straightforward fabrication. Existing literature lacks comprehensive assessments of the overall heat transfer performance of twisted tri-lobed tubes from the perspective of energy loss and irreversibility. This research aims to investigate entropy generation during turbulent water flow within twisted tri-lobed tubes, examining the influence of geometric parameters on local and average entropy production. Findings indicate that larger small circle radius (r) and straight lengths (l), coupled with smaller transition circle radius (R) and twist pitch lengths (p), result in diminished local and average heat transfer entropy production while enhancing local and average frictional entropy production, with heat transfer entropy generation dominating the overall entropy production. Additionally, with increasing Reynolds numbers, all twisted tubes demonstrate an increasing trend in average frictional entropy production, except for some cases (Case 1-Case 4) that exhibit an initial rise followed by a decline in average heat transfer entropy generation. Among the examined Reynolds range, Case 4 displays lower overall irreversibility compared to a plain tube. Following the second law of thermodynamics, Case 4 is preferred. The findings and methodology contribute to enhancing the thermodynamic evaluation of convective heat transfer in twisted tri-lobed tubes.

Random walks in a free energy landscape combining augmented molecular dynamics simulations with a dynamic graph neural network model

In this study, two approaches were applied to enhance the conformational search from molecular dynamics simulations to determine the transition states of a potential energy surface topology. The main focus is on the augmented dynamics using the swarm particle intelligence and Tsallis statistics molecular dynamics simulations of the phase transition from folding to unfolding state of a peptide in an explicit solvent environment. The transition between nodes is modelled as a random walk in a dynamic graph describing a set of basins in a free energy landscape and their pairwise relations. In this study, a dynamic graph neural networks approach is used to model the dynamic information of each free energy state as the graph evolves by observing the sequential information of edges, the time intervals between edges, and information flow. In addition, a multi-digraph approach is suggested to determine the discrete pathways of the conformation transitions between the states in that free energy surface.Besides, the role of water in the thermal and chemical denaturation of the protein is studied. This study supports the idea that the folding process is characterised by a reaction in water resulting in a reduction of the iceberg formation. Whereas unfolding by another reaction in which equilibrium is shifted towards creating iceberg states in water. In this study, the dipole-dipole correlations between the peptide and solvent are described based on an information-theoretic measure, such as local transfer entropy, to explain the role of waters in the folding/unfolding mechanism.

What Drives Bitcoin? An Approach from Continuous Local Transfer Entropy and Deep Learning Classification Models.

Bitcoin has attracted attention from different market participants due to unpredictable price patterns. Sometimes, the price has exhibited big jumps. Bitcoin prices have also had extreme, unexpected crashes. We test the predictive power of a wide range of determinants on bitcoins’ price direction under the continuous transfer entropy approach as a feature selection criterion. Accordingly, the statistically significant assets in the sense of permutation test on the nearest neighbour estimation of local transfer entropy are used as features or explanatory variables in a deep learning classification model to predict the price direction of bitcoin. The proposed variable selection do not find significative the explanatory power of NASDAQ and Tesla. Under different scenarios and metrics, the best results are obtained using the significant drivers during the pandemic as validation. In the test, the accuracy increased in the post-pandemic scenario of July 2020 to January 2021 without drivers. In other words, our results indicate that in times of high volatility, Bitcoin seems to self-regulate and does not need additional drivers to improve the accuracy of the price direction.

Modelling a multiplex brain network by local transfer entropy

This paper deals with the information transfer mechanisms underlying causal relations between brain regions under resting condition. fMRI images of a large set of healthy individuals from the 1000 Functional Connectomes Beijing Zang dataset have been considered and the causal information transfer among brain regions studied using Transfer Entropy concepts. Thus, we explored the influence of a set of states in two given regions at time t (At Bt.) over the state of one of them at a following time step (Bt+1) and could observe a series of time-dependent events corresponding to four kinds of interactions, or causal rules, pointing to (de)activation and turn off mechanisms and sharing some features with positive and negative functional connectivity. The functional architecture emerging from such rules was modelled by a directional multilayer network based upon four interaction matrices and a set of indexes describing the effects of the network structure in several dynamical processes. The statistical significance of the models produced by our approach was checked within the used database of homogeneous subjects and predicts a successful extension, in due course, to detect differences among clinical conditions and cognitive states.

What Can Local Transfer Entropy Tell Us about Phase-Amplitude Coupling in Electrophysiological Signals?

Modulation of the amplitude of high-frequency cortical field activity locked to changes in the phase of a slower brain rhythm is known as phase-amplitude coupling (PAC). The study of this phenomenon has been gaining traction in neuroscience because of several reports on its appearance in normal and pathological brain processes in humans as well as across different mammalian species. This has led to the suggestion that PAC may be an intrinsic brain process that facilitates brain inter-area communication across different spatiotemporal scales. Several methods have been proposed to measure the PAC process, but few of these enable detailed study of its time course. It appears that no studies have reported details of PAC dynamics including its possible directional delay characteristic. Here, we study and characterize the use of a novel information theoretic measure that may address this limitation: local transfer entropy. We use both simulated and actual intracranial electroencephalographic data. In both cases, we observe initial indications that local transfer entropy can be used to detect the onset and offset of modulation process periods revealed by mutual information estimated phase-amplitude coupling (MIPAC). We review our results in the context of current theories about PAC in brain electrical activity, and discuss technical issues that must be addressed to see local transfer entropy more widely applied to PAC analysis. The current work sets the foundations for further use of local transfer entropy for estimating PAC process dynamics, and extends and complements our previous work on using local mutual information to compute PAC (MIPAC).

Anatomy of information transfer for Ibis using transfer entropy and active information storage

As birds migrate in a linear formation, the frequent exchange of leader-follower in the Ibises can extend the formation flight time. In this paper, we utilize the information theory to analyze the underlying mechanism of the coordination and cooperation within the Ibises. To this end, we examine and quantify the dynamic characteristics of the information transfer in a group of Ibises at the individual level. Firstly, the local transfer entropy is employed to quantify the local dynamic information transfer during the migration of the Ibises. Therefore, we provide lucid explanations of how Ibis individually influences the group decision through information transfer to generate consistent collective behavior. The local active information storage is used to represent the memory of the Ibis. The relationship between the consistency of the collective behavior and the memory of the Ibis is obtained by calculating the mutual information between the historical state sequences and the future state of the Ibis. Finally, based on the dynamic information transfer, the relationship between the group memory and the consistent collective behavior, we effectively captured the relationship between the excitation and convergence of the Ibis’ collective behavior, and the variance of entropy is obtained.

Coupling of Conformational Switches in Calcium Sensor Unraveled with Local Markov Models and Transfer Entropy.

Proteins often have multiple switching domains that are coupled to each other and to the binding of ligands in order to realize signaling functions. Here we investigate the C2A domain of Synaptotagmin-1 (Syt-1), a calcium sensor in the neurotransmitter release machinery and a model system for the large family of C2 membrane binding domains. We combine extensive molecular dynamics (MD) simulations with Markov modeling in order to model conformational switching domains, their states, and their dependence on bound calcium ions. Then, we use transfer entropy to characterize how the switching domains are coupled via directed or allosteric mechanisms and give rise to the calcium sensing function of the protein. Our proposed switching mechanism contributes to the understanding of the neurotransmitter release machinery. Furthermore, the methodological approach we develop serves as a template to analyze conformational switching domains and the broad study of their coupling in macromolecular machines.

Performance analysis of printed circuit heat exchanger for supercritical carbon dioxide and water

Printed circuit heat exchanger could be used as pre-cooler in the supercritical carbon dioxide (S-CO2) Brayton power cycle. Due to the drastic variations of S-CO2 properties in pre-cooler, segmental design method is used to investigate the thermal-hydraulic performance of PCHE. When the heat duty is given, the local heat transfer entropy generation number has extreme value, which appears near the intersection point of the heat capacity flow rates of both sides. In addition, the local and total entropy generation mainly depend on the local and total heat transfer entropy generation, and the flow imbalance increases entropy generation, especially under conditions of the higher local heat capacity flow rate of water.

Reconstructing positive and negative couplings in Ising spin networks by sorted local transfer entropy.

We employ the sorted local transfer entropy (SLTE) to reconstruct the coupling strengths of Ising spin networks with positive and negative couplings (J_{ij}), using only the time-series data of the spins. The SLTE method is model-free in the sense that no knowledge of the underlying dynamics of the spin system is required and is applicable to a broad class of systems. Contrary to the inference of coupling from pairwise transfer entropy, our method can reliably distinguish spin pair interactions with positive and negative couplings. The method is tested for the inverse Ising problem for different J_{ij} distributions and various spin dynamics, including synchronous and asynchronous update Glauber dynamics and Kawasaki exchange dynamics. It is found that the pairwise SLTE is proportional to the pairwise coupling strength to a good extent for all cases studied. In addition, the reconstruction works well for both the equilibrium and nonequilibrium cases of the time-series data. Comparison to other inverse Ising problem approaches using mean-field equations is also discussed.

Network Structure and Dynamics of Chinese Regional Incubation

In recent years, the structure of incubation networks has attracted much attention. However, the intricacy of network has limited the discussions on its structure and dynamics. Information theory is employed as a model independent approach, which is capable of overcoming these obstacles. Information flows between different regions are calculated with the help of the transfer entropy methodology. Moreover, corresponding incubation networks are constructed over two periods. Thereafter, local transfer entropy is calculated for analysing how the structure of networks is changing with time. The results obtained are presented as follows. Firstly, the adjacent geographical location is the key factor in the formation of region incubation networks and six region incubation networks are formed in China; Secondly, network-level in different regions exhibits a serious imbalance; Finally, the policy significantly contributes to the dynamics of region incubation networks. The key findings revealed in the paper extend support to the governments and local authorities in shaping their prospective policies for not only the incubation industry, but also the entrepreneurs, and development of the region economy.

Spatiotemporal dynamics driven by the maximization of local information transfer

In this paper, a generic type of a spatially extended system, which is driven by the maximization of information transfer in each spatiotemporal point, is proposed. As an expression of the information transfer, transfer entropy is addressed, and a one-dimensional cellular system (whose state transition is governed to maximize the local transfer entropy (LTE) from interacting cells) is introduced. We first show that this system’s mechanism of state transition can be considered equivalent to a certain class of cellular automata rules with memory. The spatiotemporal dynamics of the system is then investigated to generate a wide variety of patterns, including spatiotemporal intermittency, according to the length of memory. Furthermore, the spatiotemporal patterns of states and the resulting information dynamics are statistically characterized in detail, expressing the system’s diverse nature. In particular, it is found that, within a certain condition of limited memory, even if each cell is driven to maximize the LTE, the entire system cannot reach toward its theoretical maximum value at all due to its intrinsic property, in which the system is dynamically bounding its limit on its own.

Entropy generation analysis of fully-developed turbulent heat transfer flow in inward helically corrugated tubes

The entropy generation analysis of fully-developed turbulent heat transfer flow in inward helically corrugated tubes was numerically performed by using a Reynolds stress model. The simulations were conducted for a smooth tube and five cases of corrugated tubes with Reynolds number (Re) ranging from 10,020 to 40,060 at a constant wall temperature condition. The effects of corrugation pitch and height on the flow patterns as well as local thermal and frictional entropy generation are detailed in the near wall region. The results indicate that the local heat transfer entropy generation is significantly evident at the sub-layer region and the detached vortex region, and the local thermal entropy is improved with increases in the secondary flow. Local friction entropy generation is mainly located at the windward of the corrugation and the severely turbulent fluctuation region and is mainly induced by the velocity gradient. The average friction entropy generation exhibits an exponential growth, while the average heat transfer and the total entropy generation display a linear growth trend with increased Re. The average Bejan number (Be) exhibits an exponential decline, and the minimum value can reach 0.69. From a comprehensive viewpoint, it is optimal for the Re to be lower than 30,050. When Re <20,030, higher and dense corrugations are beneficial. When 20,030 < Re <30,050, low and spare corrugations are more optimal. Besides, the case with Hl/D = 0.08 is not recommended.

Specific transfer entropy and other state-dependent transfer entropies for continuous-state input-output systems.

Since its original formulation in 2000, transfer entropy has become an invaluable tool in the toolbox of nonlinear dynamicists working with empirical data. Transfer entropy and its generalizations provide a precise definition of uncertainty and information transfer that are central to the coupled systems studied in nonlinear science. However, a canonical definition of state-dependent transfer entropy has yet to be introduced. We introduce a candidate measure, the specific transfer entropy, and compare its properties to both total and local transfer entropy. Specific transfer entropy makes possible both state- and time-resolved analysis of the predictive impact of a candidate input system on a candidate output system. We also present principled methods for estimating total, local, and specific transfer entropies from empirical data. We demonstrate the utility of specific transfer entropy and our proposed estimation procedures with two model systems, and find that specific transfer entropy provides more, and more easily interpretable, information about an input-output system compared to currently existing methods.

Methodology for Simulation and Analysis of Complex Adaptive Supply Network Structure and Dynamics Using Information Theory

Supply networks existing today in many industries can behave as complex adaptive systems making them more difficult to analyze and assess. Being able to fully understand both the complex static and dynamic structures of a complex adaptive supply network (CASN) are key to being able to make more informed management decisions and prioritize resources and production throughout the network. Previous efforts to model and analyze CASN have been impeded by the complex, dynamic nature of the systems. However, drawing from other complex adaptive systems sciences, information theory provides a model-free methodology removing many of those barriers, especially concerning complex network structure and dynamics. With minimal information about the network nodes, transfer entropy can be used to reverse engineer the network structure while local transfer entropy can be used to analyze the network structure’s dynamics. Both simulated and real-world networks were analyzed using this methodology. Applying the methodology to CASNs allows the practitioner to capitalize on observations from the highly multidisciplinary field of information theory which provides insights into CASN’s self-organization, emergence, stability/instability, and distributed computation. This not only provides managers with a more thorough understanding of a system’s structure and dynamics for management purposes, but also opens up research opportunities into eventual strategies to monitor and manage emergence and adaption within the environment.

Information transfer in a swarm of soldier crabs

Collective behavior is broadly observed in animal groups such as insect swarm, bird flock, and fish school. Both theoretical studies and field observations have investigated possible underlying principles based on local interaction among individuals in a group without global information via conductors or leaders. Information transferred among individuals would play a key role to understand it. In this study, to investigate how individual in a swarm uses information of its own past behavior or swarm mates' behavior, we analyzed behavior of soldier crabs Mictyris guinotae in terms of local active information storage and local transfer entropy.

On Thermodynamic Interpretation of Transfer Entropy

We propose a thermodynamic interpretation of transfer entropy near equilibrium, using a specialised Boltzmann’s principle. The approach relates conditional probabilities to the probabilities of the corresponding state transitions. This in turn characterises transfer entropy as a difference of two entropy rates: the rate for a resultant transition and another rate for a possibly irreversible transition within the system affected by an additional source. We then show that this difference, the local transfer entropy, is proportional to the external entropy production, possibly due to irreversibility. Near equilibrium, transfer entropy is also interpreted as the difference in equilibrium stabilities with respect to two scenarios: a default case and the case with an additional source. Finally, we demonstrated that such a thermodynamic treatment is not applicable to information flow, a measure of causal effect.

Spatiotemporal information transfer pattern differences in motor selection

Analysis of information transfer between variables in brain images is currently a popular topic, e.g. [1]. Such work typically focuses on average information transfer (i.e. transfer entropy[2]), yet the dynamics of transfer from a source to a destination can also be quantified at individual time points using the local transfer entropy (TE) [3]. This local perspective is known to reveal dynamical structure that the average cannot. We present a method to quantify local TE values in time between source and destination regions of variables in brain-imaging data, combining: a. computation of inter-regional transfer between two regions of variables (e.g. voxels) [1], with b. the local perspective of the dynamics of such transfer in time [3]. Transfer is computed over samples from all variables – there is no training in or subset selection of variables to use. We apply this method to a set of fMRI measurements where we could expect to see differences in local information transfer between two conditions at specific time steps. The fMRI data set analyzed (from [4]) contains brain activity recorded from 7 localized regions while 12 subjects (who gave informed written consent) were asked to freely decide whether to push one of two buttons (with left or right index finger), whenever they felt the urge to do so, and to press the button immediately on deciding . To our knowledge, this is the first analysis of transfer entropy on a local temporal scale in brain-imaging data (at specific time points rather than via sliding windows). Significant differences in the local TE between left and right button presses are revealed in a significant number of subjects (7) by: a. examining the difference in local TE from a single source region (e.g. pre-SMA) into left and right motor cortex respectively (e.g. see Figure ​Figure11 for subject 1); and Figure 1 Difference in TE(pre-SMA → left motor) and TE(pre-SMA → right motor) versus time after button press for subject 1. Error bars are std. err. over presses. Significant difference between left and right button press indicated by · ... b. aggregating local TE differences across 2 to 3 consecutive time steps (e.g. t=2,4 and 6 sec. after button press). Additionally, thresholding of these TE differences can decode the button push with a (statistically significant) mean of 65% accuracy across subjects. These measurements of local TE correlate well with the role of these regions in executing the motor response here [4]. We confirm that local TE can be used to reveal differences in task-based dynamical information transfer, with potential for the technique to be improved in the future.

Second law analysis of curved rectangular channels

The thermodynamic performance of the curved rectangular channels in laminar flow is numerically investigated in terms of entropy generation. The classical Navier–Stokes equations are adopted, and water is selected as the working fluid. The results show that the geometric parameters have important influences on the heat transfer performance of the channel flows. For the channels with the same cross-section, the Nusselt number increases significantly as the curvature ratio increases at the expense of slight increase of pressure drop; the dimensionless total entropy generation generally tends to reduce as Reynolds number grows, and decreases as the curvature ratio increases at the same Reynolds number. The dimensionless total entropy generation lessens with the reduction of cross-sectional area at the same Reynolds number in the channels with the same radius of curvature. Despite the rapid drop of the Bejan number, we have not found the optimal flow regime for curved rectangular channel laminar flows. The local heat transfer and fluid friction entropy generations mainly occur in the narrow region near the walls, especially the outer wall. The field synergy principle provides an alternative way to explain the heat transfer enhancement mechanism for the curved rectangular channel flows.

Local information transfer as a spatiotemporal filter for complex systems

We present a measure of local information transfer, derived from an existing averaged information-theoretical measure, namely, transfer entropy. Local transfer entropy is used to produce profiles of the information transfer into each spatiotemporal point in a complex system. These spatiotemporal profiles are useful not only as an analytical tool, but also allow explicit investigation of different parameter settings and forms of the transfer entropy metric itself. As an example, local transfer entropy is applied to cellular automata, where it is demonstrated to be a useful method of filtering for coherent structure. More importantly, local transfer entropy provides the first quantitative evidence for the long-held conjecture that the emergent traveling coherent structures known as particles (both gliders and domain walls, which have analogs in many physical processes) are the dominant information transfer agents in cellular automata.