Symbolic Transfer Entropy Research Articles

Quantitative biomarkers to predict response to clozapine treatment using resting EEG data

Clozapine is an anti-psychotic drug that is known to be effective in the treatment of patients with chronic treatment-resistant schizophrenia (TRS-SCZ), commonly estimated to be around one third of all cases. However, clinicians sometimes delay the initiation of this drug because of its adverse side-effects. Therefore, identification of predictive biomarkers of clozapine response is extremely valuable to aid on-time initiation of clozapine treatment. In this study, we develop a machine learning (ML) algorithm based on the pre-treatment electroencephalogram (EEG) data sets to predict response to clozapine treatment in TRS-SCZs, where the treatment outcome, after at least one-year follow-up is determined using the Positive and Negative Syndrome Scale (PANSS). The ML algorithm has two steps: 1) an effective connectivity named symbolic transfer entropy (STE) is applied to resting state EEG waveforms, 2) the ML algorithm is applied to STE matrix to determine whether a set of features can be found to discriminate most responder (MR) SCZ patients from least responder (LR) ones. The findings of this study revealed that the STE features could achieve an accuracy of 89.90%. This finding implies that analysis of pre-treatment EEG could contribute to our ability to distinguish MR from LR SCZs, and that the STE matrix may prove to be a promising tool for the prediction of the clinical response to clozapine.

Spontaneous Feedforward Connectivity in Electrically Stimulated Retinal Degeneration Mice.

Retinal degeneration (Rd) is a neurodegenerative disorder primarily associated with the degeneration of the retina neurons and culminates in the eventual loss of visual perception or blindness. Decrease in fronto-, parietal and occipital brain connectivity have been reported in a number of neurodegeneration diseases involving cognitive decline. However, cortical communication in the brain of retinal degeneration patients remains largely unknown and strategies to remediate observed dysfunctional brain connectivity in such instance have not be thoroughly investigated. We used rd10 mice as a model to study brain connectivity in the human retinal degeneration disease, retinitis pigmentosa. Rd10 mice with sham matched controls were electrically stimulated at varying stimulation frequencies and the consequent perturbations in feedforward brain connectivity were studied in the visual cortex and pre-frontal cortex using electrocorticography (ECoG) and normalized symbolic transfer entropy (NSTE). Contra Vcx - contra PFx feed forward connectivity significantly (p<0.05) increased in theta, alpha and beta oscillatory bands of 2 Hz and 10 Hz stimulated rd10 respectively in comparison with sham group. Also, this increase was significantly maintained even after the end of the stimulation period.

Brain Network Constancy and Participant Recognition: an Integrated Approach to Big Data and Complex Network Analysis.

With the development of big data sharing and data standardization, electroencephalogram (EEG) data are increasingly used in the exploration of human cognitive behavior. Most of the existing studies focus on the changes of human brain network topology (the number of connections, degree distribution, clustering coefficient phantom) in various cognitive behaviors. However, there has been little exploration into the steady state of multi-cognitive behaviors and the recognition of multi-participant brain networks. To solve these two problems, we used EEG data of 99 healthy participants from the PhysioBank to study multi-cognitive behaviors. Specifically, we calculated the symbolic transfer entropy (STE) between 64 electrode sequences of EEG data and constructed the brain networks of various cognitive behaviors of each participant using the directed minimum spanning tree (DMST) algorithm. We then investigated the eigenvalue spectrum of the STE matrix of each individual's cognitive behavior. The results also showed that the spectrum distributions of different cognitive states of the same participant remained relatively stable, but those of the same cognitive state of different participants varied considerably, verifying the relative stability and uniqueness of the human brain network similar to a human's fingerprint. Based on these features, we used the spectral distribution set of 99 participants of various cognitive states as the original data set and developed a spectral distribution set scoring (SDSS) method to identify the brain network participants. It was found that most labels (69.35%) of the test participant with the highest score were identical to the labeled participant. This study provided further evidence for the existence of human brain fingerprints, and furnished a new approach for dynamic identification of brain fingerprints.

Sleep Electroencephalographic Response to Respiratory Events in Patients With Moderate Sleep Apnea-Hypopnea Syndrome.

Sleep apnea–hypopnea syndrome is a common breathing disorder that can lead to organic brain injury, prevent memory consolidation, and cause other adverse mental-related complications. Brain activity while sleeping during respiratory events is related to these dysfunctions. In this study, we analyzed variations in electroencephalography (EEG) signals before, during, and after such events. Absolute and relative powers, as well as symbolic transfer entropy (STE) of scalp EEG signals, were calculated to unveil the activity of brain regions and information interactions between them, respectively. During the respiratory events, only low-frequency power increased during rapid eye movement (REM) stage (δ-band absolute and relative power) and N1 (δ- and θ-band absolute power, δ-band relative power) sleep. But absolute power increased in low- and medium-frequency bands (δ, θ, α, and σ bands), and relative power increased mainly in the medium-frequency band (α and σ bands) during stage N2 sleep. After the respiratory events, absolute power increased in all frequency bands and sleep stages, but relative power increased in medium and high frequencies. Regarding information interactions, the β-band STE decreased during and after events. In the γ band, the intrahemispheric STE increased during events and decreased afterward. Moreover, the interhemisphere STE increased after events during REM and stage N1 sleep. The EEG changes throughout respiratory events are supporting evidence for previous EEG knowledge of the impact of sleep apnea on the brain. These findings may provide insights into the influence of the sleep apnea–hypopnea syndrome on cognitive function and neuropsychiatric defects.

Symbolic transfer entropy reveals the age structure of pandemic influenza transmission from high-volume influenza-like illness data.

Existing methods to infer the relative roles of age groups in epidemic transmission can normally only accommodate a few age classes, and/or require data that are highly specific for the disease being studied. Here, symbolic transfer entropy (STE), a measure developed to identify asymmetric transfer of information between stochastic processes, is presented as a way to reveal asymmetric transmission patterns between age groups in an epidemic. STE provides a ranking of which age groups may dominate transmission, rather than a reconstruction of the explicit between-age-group transmission matrix. Using simulations, we establish that STE can identify which age groups dominate transmission even when there are differences in reporting rates between age groups and even if the data are noisy. Then, the pairwise STE is calculated between time series of influenza-like illness for 12 age groups in 884 US cities during the autumn of 2009. Elevated STE from 5 to 19 year-olds indicates that school-aged children were likely the most important transmitters of infection during the autumn wave of the 2009 pandemic in the USA. The results may be partially confounded by higher rates of physician-seeking behaviour in children compared to adults, but it is unlikely that differences in reporting rates can explain the observed differences in STE.

Information Flow between Bitcoin and Other Investment Assets

This paper studies the causal relationship between Bitcoin and other investment assets. We first test Granger causality and then calculate transfer entropy as an information-theoretic approach. Unlike the Granger causality test, we discover that transfer entropy clearly identifies causal interdependency between Bitcoin and other assets, including gold, stocks, and the U.S. dollar. However, for symbolic transfer entropy, the dynamic rise–fall pattern in return series shows an asymmetric information flow from other assets to Bitcoin. Our results imply that the Bitcoin market actively interacts with major asset markets, and its long-term equilibrium, as a nascent market, gradually synchronizes with that of other investment assets.

Teaching Ordinal Patterns to a Computer: Efficient Encoding Algorithms Based on the Lehmer Code

Ordinal patterns are the common basis of various techniques used in the study of dynamical systems and nonlinear time series analysis. The present article focusses on the computational problem of turning time series into sequences of ordinal patterns. In a first step, a numerical encoding scheme for ordinal patterns is proposed. Utilising the classical Lehmer code, it enumerates ordinal patterns by consecutive non-negative integers, starting from zero. This compact representation considerably simplifies working with ordinal patterns in the digital domain. Subsequently, three algorithms for the efficient extraction of ordinal patterns from time series are discussed, including previously published approaches that can be adapted to the Lehmer code. The respective strengths and weaknesses of those algorithms are discussed, and further substantiated by benchmark results. One of the algorithms stands out in terms of scalability: its run-time increases linearly with both the pattern order and the sequence length, while its memory footprint is practically negligible. These properties enable the study of high-dimensional pattern spaces at low computational cost. In summary, the tools described herein may improve the efficiency of virtually any ordinal pattern-based analysis method, among them quantitative measures like permutation entropy and symbolic transfer entropy, but also techniques like forbidden pattern identification. Moreover, the concepts presented may allow for putting ideas into practice that up to now had been hindered by computational burden. To enable smooth evaluation, a function library written in the C programming language, as well as language bindings and native implementations for various numerical computation environments are provided in the supplements.

Information-theoretic approach to detect directional information flow in EEG signals induced by TMS

Effective connectivity analysis has been widely applied to noninvasive recordings such as functional magnetic resonance imaging and electroencephalograms (EEGs). Previous studies have aimed to extract the causal relations between brain regions, but the validity of the derived connectivity has not yet been fully determined. This is because it is generally difficult to identify causality in the usual experimental framework based on observations alone. Transcranial magnetic stimulation (TMS) provides a framework in which a controllable perturbation is applied to a local brain region and the effect is examined by comparing the neural activity with and without this stimulation. This study evaluates two methods for effective connectivity analysis, symbolic transfer entropy (STE) and vector autoregression (VAR), by applying them to TMS-EEG data. In terms of the consistency of results from different experimental sessions, STE is found to yield robust results irrespective of sessions, whereas VAR produces less correlation between sessions. Furthermore, STE preferentially detects the directional information flow from the TMS target. Taken together, our results suggest that STE is a reliable method for detecting the effect of TMS, implying that it would also be useful for identifying neural activity during cognitive tasks and resting states.

Adaptive symbolic transfer entropy and its applications in modeling for complex industrial systems.

Directed coupling between variables is the foundation of studying the dynamical behavior of complex systems. We propose an adaptive symbolic transfer entropy (ASTE) method based on the principle of equal probability division. First, the adaptive kernel density method is used to obtain an accurate probability density function for an observation series. Second, the complete phase space of the system can be obtained by using the multivariable phase space reconstruction method. This provides common parameters for symbolizing a time series, including delay time and embedding dimension. Third, an optimization strategy is used to select the appropriate symbolic parameters of a time series, such as the symbol set and partition intervals, which can be used to convert the time series to a symbol sequence. Then the transfer entropy between the symbolic sequences can be carried out. Finally, the proposed method is analyzed and validated using the chaotic Lorenz system and typical complex industrial systems. The results show that the ASTE method is superior to the existing transfer entropy and symbolic transfer entropy methods in terms of measurement accuracy and noise resistance, and it can be applied to the network modeling and performance safety analysis of complex industrial systems.

Detecting coupling directions with transcript mutual information: A comparative study

<p style='text-indent:20px;'>Causal relationships are important to understand the dynamics of coupled processes and, moreover, to influence or control the effects by acting on the causes. Among the different approaches to determine cause-effect relationships and, in particular, coupling directions in interacting random or deterministic processes, we focus in this paper on information-theoretic measures. So, we study in the theoretical part the difference between directionality indicators based on transfer entropy as well as on its dimensional reduction via transcripts in algebraic time series representations. In the applications we consider specifically the lowest dimensional case, i.e., 3-dimensional transfer entropy, which is currently one of the most popular causality indicators, and the (2-dimensional) mutual information of transcripts. Needless to say, the lower dimensionality of the transcript-based indicator can make a difference in practice, where datasets are usually small. To compare numerically the performance of both directionality indicators, synthetic data (obtained with random processes) and real world data (in the form of biomedical recordings) are used. As happened in previous related work, we found again that the transcript mutual information performs as good as, and in some cases even better than, the lowest dimensional binned and symbolic transfer entropy, the symbols being ordinal patterns.

Stroke causes a transient imbalance of interhemispheric information flow in EEG during non-REM sleep

ObjectiveLarge-scale connectivity, especially interhemispheric connections, plays a crucial role for recovery after stroke. Here we used methods from information theory to characterize interhemispheric information flow in wake- and sleep-EEG after cerebral ischemia. Methods34 patients with unilateral ischemic stroke were included. Symbolic Transfer Entropy (STE) was applied between bipolar EEG signals on the left and the right cerebral hemisphere during polysomnographic recordings in the acute phase and 3 months after stroke. ResultsIn the acute phase, we found a sleep stage-dependent preferred interhemispheric asymmetry: during non-REM sleep the information flow was predominantly directed from the contralesional toward the ipsilesional hemisphere. This effect was greatly reduced in a follow-up recording 3 months after stroke onset. ConclusionOur findings are consistent with functional imaging studies showing a transient hyperactivity of contralesional areas after stroke. We conclude that STE is a robust method for detecting post-stroke connectivity reorganizations, and that sleep stages have to be taken into account when assessing functional connectivity. SignificanceEEG is more widely available than functional MRI. Future studies will have to confirm whether EEG derived STE can be useful in a clinical setting during rehabilitation after stroke.

Consciousness Indexing and Outcome Prediction with Resting-State EEG in Severe Disorders of Consciousness.

We applied the following methods to resting-state EEG data from patients with disorders of consciousness (DOC) for consciousness indexing and outcome prediction: microstates, entropy (i.e. approximate, permutation), power in alpha and delta frequency bands, and connectivity (i.e. weighted symbolic mutual information, symbolic transfer entropy, complex network analysis). Patients with unresponsive wakefulness syndrome (UWS) and patients in a minimally conscious state (MCS) were classified into these two categories by fitting and testing a generalised linear model. We aimed subsequently to develop an automated system for outcome prediction in severe DOC by selecting an optimal subset of features using sequential floating forward selection (SFFS). The two outcome categories were defined as UWS or dead, and MCS or emerged from MCS. Percentage of time spent in microstate D in the alpha frequency band performed best at distinguishing MCS from UWS patients. The average clustering coefficient obtained from thresholding beta coherence performed best at predicting outcome. The optimal subset of features selected with SFFS consisted of the frequency of microstate A in the 2-20Hz frequency band, path length obtained from thresholding alpha coherence, and average path length obtained from thresholding alpha coherence. Combining these features seemed to afford high prediction power. Python and MATLAB toolboxes for the above calculations are freely available under the GNU public license for non-commercial use ( https://qeeg.wordpress.com ).

Time series analysis of the Antarctic Circumpolar Wave via symbolic transfer entropy

An attempt to interpret a large-scale climate phenomenon in the Southern Ocean (SO), the Antarctic Circumpolar Wave (ACW), has been made using an information entropy method, symbolic transfer entropy (STE). Over the areas of 50–60∘S latitude belt, information flow for four climate variables, sea surface temperature (SST), sea–ice edge (SIE), sea level pressure (SLP) and meridional wind speed (MWS) is examined. We found a tendency that eastward flow of information is preferred only for oceanic variables, which is a main characteristic of the ACW, an eastward wave making a circuit around the Antarctica. Since the ACW is the coherent pattern in both ocean and atmosphere it is reasonable to infer that the tendency reflects the Antarctic Circumpolar Current (ACC) encircling the Antarctica, rather than an evidence of the ACW. We observed one common feature for all four variables, a strong information flow over the area of the eastern Pacific Ocean, which suggest a signature of El Nino Southern Oscillation (ENSO).

Electroencephalogram-Electromyography Coupling Analysis in Stroke Based on Symbolic Transfer Entropy.

The coupling strength between electroencephalogram (EEG) and electromyography (EMG) signals during motion control reflects the interaction between the cerebral motor cortex and muscles. Therefore, neuromuscular coupling characterization is instructive in assessing motor function. In this study, to overcome the limitation of losing the characteristics of signals in conventional time series symbolization methods, a variable scale symbolic transfer entropy (VS-STE) analysis approach was proposed for corticomuscular coupling evaluation. Post-stroke patients (n = 5) and healthy volunteers (n = 7) were recruited and participated in various tasks (left and right hand gripping, elbow bending). The proposed VS-STE was employed to evaluate the corticomuscular coupling strength between the EEG signal measured from the motor cortex and EMG signal measured from the upper limb in both the time-domain and frequency-domain. Results showed a greater strength of the bi-directional (EEG-to-EMG and EMG-to-EEG) VS-STE in post-stroke patients compared to healthy controls. In addition, the strongest EEG–EMG coupling strength was observed in the beta frequency band (15–35 Hz) during the upper limb movement. The predefined coupling strength of EMG-to-EEG in the affected side of the patient was larger than that of EEG-to-EMG. In conclusion, the results suggested that the corticomuscular coupling is bi-directional, and the proposed VS-STE can be used to quantitatively characterize the non-linear synchronization characteristics and information interaction between the primary motor cortex and muscles.

Multivariate Symbolic Transfer Entropy Analysis of Different Age Groups

The life activities of the human body are closely related to age, and the life process is constantly associated with the entropy change. In this paper, multivariate symbolic transfer entropy algorithm was proposed based on traditional symbolic transfer entropy, then we also adopted the dynamic adaptive method to symbolize the time series. This algorithm was used to study the relationship between multivariate symbolic transfer entropy and age. Experiments showed that the value of multivariate symbolic transfer entropy of middle-aged man is greater than that of the young. The hypothesis test can prove that the multivariate symbolic transfer entropy shows a significant difference in age. Thus, multivariate symbolic transfer entropy algorithm can provide a new method for the study of other physiological signals.

An information transfer based novel framework for fault root cause tracing of complex electromechanical systems in the processing industry

Abstract As one of the most important approaches for analyzing the mechanism of fault pervasion, fault root cause tracing is a powerful and useful tool for detecting the fundamental causes of faults so as to prevent any further propagation and amplification. Focused on the problems arising from the lack of systematic and comprehensive integration, an information transfer-based novel data-driven framework for fault root cause tracing of complex electromechanical systems in the processing industry was proposed, taking into consideration the experience and qualitative analysis of conventional fault root cause tracing methods. Firstly, an improved symbolic transfer entropy method was presented to construct a directed-weighted information model for a specific complex electromechanical system based on the information flow. Secondly, considering the feedback mechanisms in the complex electromechanical systems, a method for determining the threshold values of weights was developed to explore the disciplines of fault propagation. Lastly, an iterative method was introduced to identify the fault development process. The fault root cause was traced by analyzing the changes in information transfer between the nodes along with the fault propagation pathway. An actual fault root cause tracing application of a complex electromechanical system is used to verify the effectiveness of the proposed framework. A unique fault root cause is obtained regardless of the choice of the initial variable. Thus, the proposed framework can be flexibly and effectively used in fault root cause tracing for complex electromechanical systems in the processing industry, and formulate the foundation of system vulnerability analysis and condition prediction, as well as other engineering applications.

Study of the cross-market effects of Brexit based on the improved symbolic transfer entropy GARCH model-An empirical analysis of stock-bond correlations.

In this paper, we study the cross-market effects of Brexit on the stock and bond markets of nine major countries in the world. By incorporating information theory, we introduce the time-varying impact weights based on symbolic transfer entropy to improve the traditional GARCH model. The empirical results show that under the influence of Brexit, flight-to-quality not only commonly occurs between the stocks and bonds of each country but also simultaneously occurs among different countries. We also find that the accuracy of the time-varying symbolic transfer entropy GARCH model proposed in this paper has been improved compared to the traditional GARCH model, which indicates that it has a certain practical application value.

Interaction analysis–based information modeling of complex electromechanical systems in the processing industry

Information modeling for complex electromechanical systems in the processing industry is the foundation for system vulnerability analysis, failure propagation mechanism, and fault root cause tracing driven by data analysis. Focusing on the difficulties in information modeling for complex electromechanical systems, a new approach based on interaction analysis and a general framework for information modeling of complex electromechanical systems are proposed. First, the basic structures of the information model are defined based on monitoring data. Second, an improved symbolic transfer entropy method with procedures for optimizing the number of symbols, phase space reconstruction, binary encoding, and decimal decoding are proposed to detect the interaction direction and quantify the interaction strength between different monitoring variables. Third, some optional methods are introduced to simplify and modify models. Finally, a directed-weighted information model for a specific complex electromechanical system is constructed based on information flow. An actual information modeling application of a complex electromechanical system is used to demonstrate the proposed method and compare it with existing methods. This new approach can handle general information modeling problems and overcome the drawbacks of existing methods since all the monitoring variables are used to improve integrity of the model, and prior knowledge about the physical structure and key points’ selection are not required. A unique and complete information model is obtained regardless of the choice of the initial variable. Thus, the proposed method can be flexibly and effectively used in information modeling for complex electromechanical systems in the processing industry and formulate the foundation of system vulnerability analysis, failure propagation mechanism, and fault root cause tracing methodology, as well as other engineering applications.

Propofol-induced sedation diminishes the strength of frontal-parietal-occipital EEG network.

The level of conscious experience can be effectively and reversibly altered by the administration of sedative agents. Several studies attempted to explore the variations in frontal-parietal network during propofol-induced sedation. However, contradictory outcomes warrant further investigations. In this study, we implemented the Neural Gas algorithm-based delay symbolic transfer entropy (NG-dSTE) for investigation of frontal-parietal-occipital (F-P-O) network using scalp EEG signals recorded during altered levels of consciousness. Our results show significant disruption of the F-P-O network during mild and moderate levels of propofol sedation. In particular, the interaction between frontal and parietal-occipital region is highly disturbed. Moreover, we found measurable effect of sedation on local interactions in the frontal network whereas parietal-occipital network experienced least variations. The results support the conclusion that the connectivity based features can be utilized as reliable biomarker for assessment of sedation levels effectively.

Analysis of Sleep Staging Based on Multivariate Symbolic Transfer Entropy

Physiological electrical signals such as ECG, EEG and EMG during sleep contain a lot of physiological information. The analysis of these signals can provide effective advice for the diagnosis of sleep staging or sleep disorders. The symbol transfer entropy algorithm is applied to study the interaction information of physiological signals. However, because the traditional symbol transfer entropy algorithm only focuses on the relationship between one or two variables, this paper used the multivariate symbolic transfer entropy based on the traditional symbol transfer entropy to consider the coupling relationship between multiple variables. When dividing the time series, we used two methods respectively: static partitioning and dynamic adaptive. By comparing the multivariate symbolic transfer entropy values in the awake and sleep periods of different subjects, we can get that the multivariate symbol shift entropy of the awake period is significantly higher than that of the sleep subjects, although the entropy values are different from those of the subjects. And there is a significant difference between the two by using T test, which is consistent with the theory that the degree of brain disorder decreases and the entropy decreases when the sleep is deepen. The multivariate symbolic transfer entropy algorithm is effective in distinguishing human awake period from sleep period and can provide an effective way for other physiological signals research.