Multivariate Transfer Entropy Research Articles

A hybrid framework for asphalt pavement rutting prediction modeling and influencing factors analysis based on multilevel wavelet decomposition and transfer entropy

Rutting prediction model is of great significance for asphalt pavement quality assessment and road management. As a typical complex system, the rutting evaluation of asphalt pavement is causaed by multiple factors. Meanwhile, the observation of rutting and each factors show different characterizes on different frequency bands. Therefore, the accuracy of predictive model depends on the time series decomposition and the selection of input factors. In light of this, this paper proposes a novel time series forecasting framework for rutting prediction, which consists of multilevel discrete wavelet decomposition (MDWD) based data processing module, multivariate transfer entropy (mTE) based feature selection module, and time series forecasting module. In the MDWD based data processing module, rutting and its influencing factors time series are decomposed into low-frequency band and high-frequency band by multilevel wavelet decomposition, which represent trend component and fluctuations component of original series, respectively. Each component is fed into mTE-based feature selection module, the statistically significant casual factors are selected by multivariate transfer entropy, which provides a novel insight for the effect of influence factors on rutting. Finally, the components in different frequency bands are predicted with the selected causal factors, and the predicted rutting is generated by wavelet reconstruction. To validate the performance of the proposed framework, we conduct some experiments with RIOHTrack data set. The performance comparison results with other prediction models demonstrate that the proposed framework is effective. Ablation study is set up for each module in the proposed framework, which shows that each module in the framework is indispensable.

Rutting prediction and analysis of influence factors based on multivariate transfer entropy and graph neural networks

The Rutting prediction model is an essential element of efficient pavement management systems. Accuracy of commonly used predictive model necessitates knowledge of the input parameters that was incorporated and local calibration of the model coefficients. In this paper, a novel rutting prediction model based on multivariate transfer entropy and graph neural networks is proposed for incorporating a limited number of observable inputs, which can accommodate with sufficient prediction performance and generalization to a variety of complex pavement design structure data. The multivariate transfer entropy based graph representation is able to find the significant causality between variables and rutting. The influence factor analysis results confirm the high influence of temperature and vehicle axle load. Several experiments are set up on the Research Institute of Highway Ministry of Transport track (RIOHTrack) dataset for the comparison between the proposed model and the state-of-art prediction models. The result demonstrates that the proposed model is more accurate and robust compared to existing methods on the rutting prediction task.

Research on the MEG of Depression Patients Based on Multivariate Transfer Entropy.

The pathogenesis of depression is complex, and the current means of medical diagnosis is single. Patients with severe depression may even have great physical pain and suicidal tendencies. Magnetoencephalography (MEG) has the characteristics of ultrahigh spatiotemporal resolution and safety. It is a good medical means for the diagnosis of depression. In this paper, multivariate transfer entropy algorithm is used to study MEG of depression. In this paper, the subjects are divided into the same brain region and the multichannel combination between different brain regions, and the multivariate transfer entropy of patients with depression and healthy controls under different EEG signal frequency bands is calculated. Finally, the significant difference between the two groups of experimental samples is verified by the results of independent sample t-test. The experimental results show that for the same combination of brain channels, the multivariate transfer entropy in the depression group is generally lower than that in the healthy control group, and the difference is the best in γ frequency band and the largest in the frontal region.

Efficient markets are more connected: An entropy-based analysis of the energy, industrial metal and financial markets

This is the first article to jointly study the efficiency and connectedness of commodity markets, using the novel method combination of fuzzy entropy and multivariate transfer entropy analysis. We examine this issue in the nexus of energy, industrial metals and financial markets. We identify the existence of a relationship between market efficiency and connectedness, which opens a new direction for investigating the sources of market connectedness. The results indicate that the efficient markets process more information and are more connected in the system. Moreover, efficient markets are net transmitters of information to the less efficient markets. Stronger connectedness does not necessarily exist within the same market sector but emerges among the efficient markets irrespective of their sector categorization. The relationship between market efficiency and connectedness is more pronounced during extreme events. During the turbulence of COVID-19, there are stronger connectedness and higher information spillover from the efficient to less efficient markets. Specifically, we find the industrial metals are on average the most efficient sector, followed by the energy and then the financial markets. The relatively more efficient natural gas and industrial metal markets exhibit strong connectedness and transmit information to the less efficient crude oil and financial markets. These findings imply that the market connectedness can be the result of information dissemination and alleviate the concerns that higher connectedness may compromise the price discovery mechanism.

A strategy of improving indoor air temperature prediction in HVAC system based on multivariate transfer entropy

The prediction for indoor temperature as an essential reference for system tuning has attracted great concern in the HVAC field. The existing researches mostly focused on transplanting novel machine learning algorithms, especially recurrent neural network (RNN), from the neuro-linguistic programming (NLP) like text generation and speech recognition into HVAC systems. But unlike semantics, the large-scale data processing during the training phase requires the high-speed processor and large memory, which restricts their online application in HVAC systems. In this work, we develop a strategy of improving gated RNN prediction for indoor temperature by estimating time delays of HVAC systems. The suggested strategy can lighten the computational burden and decrease memory capacity to favor online prediction and control, where the traceback range of error is narrowed down to the nearest-neighbors delay periods and the parameter learning time is reduced to about 2% of the original. Among them, we introduce transfer entropy (TE) into the HVAC field to estimate its time delays by mining monitoring data. Compared with correlation coefficients, it can filter redundant information between variables and discern the nonlinearity. Subordinately, we reconstruct time series by embedding time delays as inputs of the prediction model, and clip gradients within backpropagation through time (BPTT) with multivariate TE index, which promote the correlation between inputs and output and absorb the most valuable information in the parameter learning. Lastly, the proposed methods are validated with simulated and real time series. This work is enlightening and instructive to improving time series prediction in HVAC systems.

A model-free method for identifying time-delay characteristics of HVAC system based on multivariate transfer entropy

The time delays in the HVAC system are common, which seriously affect time series prediction and pre-control command implementation, so identifying them is promising research. The existing methods of identifying time-delays mainly focus on establishing physical or dynamical models first and then conducting control variable experiments, which don't facilitate engineering. Therefore, we develop a model-free identification method rooted in an information-theoretic framework by introducing transfer entropy (TE) into the HVAC field. The suggested multivariate TE method can pick out each time-delay characteristic by mining monitoring data. Compared with correlation coefficients, it can filter redundant information between variables and discern the nonlinearity. Among them, to estimate multivariate TE well, a kernel estimator is improved. It performs the precise detection ability, low computational burden and strong robustness, compared with the original and k-nearest-neighbor (KNN) estimator. Besides, for an unacquainted HVAC system, a hierarchical optimization algorithm combining the Nash-optimization algorithm with a second-order oscillatory particle swarm optimization (SOPSO) algorithm is proposed to identify its time delays, where the accuracy and time cost are improved. Lastly, the above-mentioned methods are validated with simulated and real time series. This work is enlightening and has a further reference to identifying time delays in HVAC systems.

Nonlinear directed information flow estimation for fNIRS brain network analysis based on the modified multivariate transfer entropy

Due to the specific advantages of functional near-infrared spectroscopy (fNIRS), fNIRS-based brain network research, especially directed causal coupling analysis, is emerging gradually. The transfer entropy (TE) method has proved its superiority and effectiveness in identifying directed information flow in the brain network, which helps us further understand the neurodynamic mechanisms. However, there remain three inherent defects in TE: (1) lack of multivariable effect, (2) poor robustness, and (3) dimensional curse in the high dimensional system. This study introduces a novel multivariate transfer entropy based on the nonuniform embedding, abbreviated “MTENUE,” to solve the above limits and construct the whole brain's directed coupling network. The MTENUE thoroughly considers the potential influence of remaining confounding variables in conditional mutual information estimation, guaranteeing its applicability in a multivariable system. By combining low-dimensional approximation and search strategy of particle swarm optimization, a novel nonuniform embedding (NUE) algorithm was introduced to establish the MTENUE methodology. Using a well-known Hénon map model, we carried out a series of numerical simulations to verify the performances of MTENUE and compared them with the traditional TE methods. The numerical simulations prove that the proposed MTENUE allows directional coupling estimation for a high dimensional, nonlinear, nonstationary system. The MTENUE achieves the best robustness to data length and noise. Meanwhile, it presents the best linearity for coupling parameters “C,” with its higher fitting coefficients of determination (R-square) than other methods. Moreover, by avoiding the dimensional curse, the MTENUE achieves the highest accuracy and specificity, which are higher than 98%; in contrast, the traditional methods are less than 80%. Finally, we applied the proposed method to the actual fNIRS research. The results demonstrate a typical information flow pattern changes during the working memory tasks, which is expected to provide new insights into the underlying neural dynamics in the cognitive process.

Data-driven identification model for associated fault propagation path

In this paper, a data-driven identification model of the associated fault propagation path is proposed. Different from traditional fault diagnosis methods, the alternative approach is focused more on identifying the fault propagation path. Firstly, the KPCA method is used for fault detection. Then, a new transfer entropy method is proposed to construct the causality diagram. Finally, a kernel extreme learning machine-based search approach is proposed for fault propagation path identification. To demonstrate the effectiveness and applicability, the proposed model is applied to the Tennessee-Eastman Process and the real telemetry data of the satellite in orbit. Experimental results show that the number of causal edges obtained by the proposed method is only 12.62% of the traditional transfer entropy method, and 24.45% of the multivariate transfer entropy method.

Detection and analysis of real-time anomalies in large-scale complex system

For data-driven anomaly detection, it is difficult to model a prediction model with high accuracy and sensitivity to anomalous states. In order to solve the above problems, this paper proposes an improved multivariate Transfer Entropy method to judge the physical causality between high-dimensional time series. According to the causality, combined with the LSTM (Long Short-Term Memory) model, the CF-LSTM (causality features-LSTM) model is proposed. The CF-LSTM model uses the causality features of the parameter to make predictions. Compared with other models, the CF-LSTM model improves the prediction accuracy and it is sensitive to anomalies. Based on this, an anomaly detection algorithm is proposed. Case analysis based on the actual data shows that the detection precision, recall, and F1-score (a measure of classification problem) of this method are improved compared with other anomaly detection models, which illustrates the effectiveness of this method.

Multiscale multivariate transfer entropy and application to functional corticocortical coupling

Objective. Complex biological systems consist of multi-level mechanism in terms of within- and cross-subsystems correlations, and they are primarily manifested in terms of connectivity, multiscale properties, and nonlinearity. Existing studies have each only explored one aspect of the functional corticocortical coupling (FCCC), which has some limitations in portraying the complexity of multivariable systems. The present study investigated the direct interactions of brain networks at multiple time scales. Approach. We extended the multivariate transfer entropy (MuTE) method and proposed a novel method, named multiscale multivariate transfer entropy (MSMVTE), to explore the direct interactions of brain networks across multiple time scale. To verify this aim, we introduced three simulation models and compared them with multiscale transfer entropy (MSTE) and MuTE methods. We then applied MSMVTE method to analyze FCCC during a unilateral right-hand steady-state force task. Main results. Simulation results showed that the MSMVTE method, compared with MSTE and MuTE methods, better detected direct interactions and avoid the spurious effects of indirect relationships. Further analysis of experimental data showed that the connectivity from left premotor/sensorimotor cortex to right premotor/sensorimotor cortex was significantly higher than that of opposite directionality. Furthermore, the connectivities from central motor areas to both sides of premotor/sensorimotor areas were higher than those of opposite directionalities. Additionally, the maximum coupling strength was found to occur at a specific scale (3–10). Significance. Simulation results confirmed the effectiveness of the MSMVTE method to describe direct relationships and multiscale characteristics in complex systems. The enhancement of FCCC reflects the interaction of more extended activation in cortical motor regions. Additionally, the neurodynamic process of brain depends not only on emergent behavior at small scales, but also on the constraining effects of the activity at large scales. Taken together, our findings provide a basis for better understanding dynamics in brain networks.

Inferring network properties from time series using transfer entropy and mutual information: Validation of multivariate versus bivariate approaches

Functional and effective networks inferred from time series are at the core of network neuroscience. Interpreting properties of these networks requires inferred network models to reflect key underlying structural features. However, even a few spurious links can severely distort network measures, posing a challenge for functional connectomes. We study the extent to which micro- and macroscopic properties of underlying networks can be inferred by algorithms based on mutual information and bivariate/multivariate transfer entropy. The validation is performed on two macaque connectomes and on synthetic networks with various topologies (regular lattice, small-world, random, scale-free, modular). Simulations are based on a neural mass model and on autoregressive dynamics (employing Gaussian estimators for direct comparison to functional connectivity and Granger causality). We find that multivariate transfer entropy captures key properties of all network structures for longer time series. Bivariate methods can achieve higher recall (sensitivity) for shorter time series but are unable to control false positives (lower specificity) as available data increases. This leads to overestimated clustering, small-world, and rich-club coefficients, underestimated shortest path lengths and hub centrality, and fattened degree distribution tails. Caution should therefore be used when interpreting network properties of functional connectomes obtained via correlation or pairwise statistical dependence measures, rather than more holistic (yet data-hungry) multivariate models.

Sleep-Dependent Directional Interactions of the Central Nervous System-Cardiorespiratory Network.

We investigated the nature of interactions between the central nervous system (CNS) and the cardiorespiratory system during sleep. Overnight polysomnography recordings were obtained from 33 healthy individuals. The relative spectral powers of five frequency bands, three ECG morphological features and respiratory rate were obtained from six EEG channels, ECG, and oronasal airflow, respectively. The synchronous feature series were interpolated to 1 Hz to retain the high time-resolution required to detect rapid physiological variations. CNS-cardiorespiratory interaction networks were built for each EEG channel and a directionality analysis was conducted using multivariate transfer entropy. Finally, the difference in interaction between Deep, Light, and REM sleep (DS, LS, and REM) was studied. Bidirectional interactions existed in central-cardiorespiratory networks, and the dominant direction was from the cardiorespiratory system to the brain during all sleep stages. Sleep stages had evident influence on these interactions, with the strength of information transfer from heart rate and respiration rate to the brain gradually increasing with the sequence of REM, LS, and DS. Furthermore, the occipital lobe appeared to receive the most input from the cardiorespiratory system during LS. Finally, different ECG morphological features were found to be involved with various central-cardiac and cardiac-respiratory interactions. These findings reveal detailed information regarding CNS-cardiorespiratory interactions during sleep and provide new insights into understanding of sleep control mechanisms. Our approach may facilitate the investigation of the pathological cardiorespiratory complications of sleep disorders.

Satellite On-Orbit Anomaly Detection Method Based on a Dynamic Threshold and Causality Pruning

It is difficult for existing deep learning-based satellite on-orbit anomaly detection methods to define the residual-based detection threshold and identify false anomalies. To solve the above problems, this paper proposes both a detection threshold determination and dynamic correction method and a causality-based false anomaly identification and pruning method. We use the GRU (Gated Recurrent Unit) to model and predict the telemetry parameters to obtain the residual vector; determine and dynamically correct the threshold according to the prescribed false positive rate; propose an improved multivariate transfer entropy method to identify the causal relationships between the telemetry parameters; and, based on the causality, determine whether the detected parameter anomalies are false. Experiments show that the precision, recall, and F1-score of the method proposed in this paper are superior to the current typical method, and the false positive rate is significantly reduced, demonstrating the effectiveness of the proposed method.

Network Analysis of Multivariate Transfer Entropy of Cryptocurrencies in Times of Turbulence.

We investigate the effects of the recent financial turbulence of 2020 on the market of cryptocurrencies taking into account the hourly price and volume of transactions from December 2019 to April 2020. The data were subdivided into time frames and analyzed the directed network generated by the estimation of the multivariate transfer entropy. The approach followed here is based on a greedy algorithm and multiple hypothesis testing. Then, we explored the clustering coefficient and the degree distributions of nodes for each subperiod. It is found the clustering coefficient increases dramatically in March and coincides with the most severe fall of the recent worldwide stock markets crash. Further, the log-likelihood in all cases bent over a power law distribution, with a higher estimated power during the period of major financial contraction. Our results suggest the financial turbulence induce a higher flow of information on the cryptocurrency market in the sense of a higher clustering coefficient and complexity of the network. Hence, the complex properties of the multivariate transfer entropy network may provide early warning signals of increasing systematic risk in turbulence times of the cryptocurrency markets.

Investigation of brain-heart network during sleep.

Interactions between brain and heart play an important role for sleep quality and control. However, the influence mechanism was still unclear. This study aimed to further investigate this mechanism according to build an information transfer network of brain-heart coupling. This study included 24 healthy individuals and both of them underwent overnight polysomnography. The relative spectral powers of five frequency bands and the high frequency power of heart rate variability were extracted from six electroencephalogram (EEG) channels and electrocardiography (ECG) respectively. For each EEG channel, brain-heart interaction networks were built and a directionality analysis was conducted by using multivariate transfer entropy. Results revealed the bidirectionality of information transfer between brain and heart during sleep, and the information was dominantly transfer from heart to brain. The information transfer strength between brain and heart were significantly stronger than which between frequency bands in each EEG channels. Besides, the frequency bands and EEG channels had evident influence on these interactions. This study exposed more detailed characteristics of brain-heart interaction, which will facilitate the future study about the sleep control and the diagnose of sleep related disease.

Transfer Entropy as a Measure of Brain Connectivity: A Critical Analysis With the Help of Neural Mass Models.

Objective: Assessing brain connectivity from electrophysiological signals is of great relevance in neuroscience, but results are still debated and depend crucially on how connectivity is defined and on mathematical instruments utilized. Aim of this work is to assess the capacity of bivariate Transfer Entropy (TE) to evaluate connectivity, using data generated from simple neural mass models of connected Regions of Interest (ROIs).Approach: Signals simulating mean field potentials were generated assuming two, three or four ROIs, connected via excitatory or by-synaptic inhibitory links. We investigated whether the presence of a statistically significant connection can be detected and if connection strength can be quantified.Main Results: Results suggest that TE can reliably estimate the strength of connectivity if neural populations work in their linear regions, and if the epoch lengths are longer than 10 s. In case of multivariate networks, some spurious connections can emerge (i.e., a statistically significant TE even in the absence of a true connection); however, quite a good correlation between TE and synaptic strength is still preserved. Moreover, TE appears more robust for distal regions (longer delays) compared with proximal regions (smaller delays): an approximate a priori knowledge on this delay can improve the procedure. Finally, non-linear phenomena affect the assessment of connectivity, since they may significantly reduce TE estimation: information transmission between two ROIs may be weak, due to non-linear phenomena, even if a strong causal connection is present.Significance: Changes in functional connectivity during different tasks or brain conditions, might not always reflect a true change in the connecting network, but rather a change in information transmission. A limitation of the work is the use of bivariate TE. In perspective, the use of multivariate TE can improve estimation and reduce some of the problems encountered in the present study.

Measuring information transfer by dispersion transfer entropy

In this paper, we propose a novel technique, called dispersion transfer entropy(DTE), to determine the information transfer and causal relation in the analysis of complex systems. Symbolization is used to solve the computational burden and noise sensitivity. To deal with the two major issues in symbolization, generating partition and information loss, we use the Ragwitz criterion to dynamically select parameters and utilize dispersion pattern to keep influential information. Moreover, we extend DTE into the multivariate system and propose dispersion multivariate transfer entropy(DMTE), dispersion multivariate transfer entropy curve(DMTEC) and dispersion partial transfer entropy(DPTE). DMTE greatly weakens the influence of synchronicity and similarity in data on information transfer detection, which is a breakthrough in solving the limitation of transfer entropy. DMTEC shows the evolution of causal relation over time and DPTE measures the direct causal effect between systems. These statistics can be combined to obtain a more comprehensive and accurate measurement of causality for multivariable systems. Also, we apply these methods to simulation data as well as stock markets to verify the effectiveness of our methods.

Research on the MEG of depression patients based on multivariate TE partial information decomposition

Transfer entropy (TE) has been broadly used in the field of neurosciences. In this paper, the partial information decomposition algorithm is employed to decompose multivariate TE into synergistic, redundant and unique parts. In this work, the synergistic part is believed as more suitable as the computation method. We recorded the magnetoencephalogram (MEG) data of 6 subjects with depression and 13 normal subjects under different emotional stimulations, and studied the coupling between multiple symmetric channels in the frontal area in the brain of subject. The experimental results show that under different emotional stimulations, normal people present significant difference from the depression patients, especially in the right frontal area. Furthermore, under negative emotional stimulation, the difference in synergistic value between normal people and depression patients is smaller. The synergistic value of depression patients has become bigger, which indicates that the brain complexity of depression patients has grown, and their brain activities have increased.

Measuring the Non-linear Directed Information Flow in Schizophrenia by Multivariate Transfer Entropy.

People living with schizophrenia (SCZ) experience severe brain network deterioration. The brain is constantly fizzling with non-linear causal activities measured by electroencephalogram (EEG) and despite the variety of effective connectivity methods, only few approaches can quantify the direct non-linear causal interactions. To circumvent this problem, we are motivated to quantitatively measure the effective connectivity by multivariate transfer entropy (MTE) which has been demonstrated to be able to capture both linear and non-linear causal relationships effectively. In this work, we propose to construct the EEG effective network by MTE and further compare its performance with the Granger causal analysis (GCA) and Bivariate transfer entropy (BVTE). The simulation results quantitatively show that MTE outperformed GCA and BVTE under varied signal-to-noise conditions, edges recovered, sensitivity, and specificity. Moreover, its applications to the P300 task EEG of healthy controls (HC) and SCZ patients further clearly show the deteriorated network interactions of SCZ, compared to that of the HC. The MTE provides a novel tool to potentially deepen our knowledge of the brain network deterioration of the SCZ.

Altered Causal Coupling Pathways within the Central-Autonomic-Network in Patients Suffering from Schizophrenia.

The multivariate analysis of coupling pathways within physiological (sub)systems focusing on identifying healthy and diseased conditions. In this study, we investigated a part of the central-autonomic-network (CAN) in 17 patients suffering from schizophrenia (SZO) compared to 17 age–gender matched healthy controls (CON) applying linear and nonlinear causal coupling approaches (normalized short time partial directed coherence, multivariate transfer entropy). Therefore, from all subjects continuous heart rate (successive beat-to-beat intervals, BBI), synchronized maximum successive systolic blood pressure amplitudes (SYS), synchronized calibrated respiratory inductive plethysmography signal (respiratory frequency, RESP), and the power PEEG of frontal EEG activity were investigated for 15 min under resting conditions. The CAN revealed a bidirectional coupling structure, with central driving towards blood pressure (SYS), and respiratory driving towards PEEG. The central-cardiac, central-vascular, and central-respiratory couplings are more dominated by linear regulatory mechanisms than nonlinear ones. The CAN showed significantly weaker nonlinear central-cardiovascular and central-cardiorespiratory coupling pathways, and significantly stronger linear central influence on the vascular system, and on the other hand significantly stronger linear respiratory and cardiac influences on central activity in SZO compared to CON, and thus, providing better understanding of the interrelationship of central and autonomic regulatory mechanisms in schizophrenia might be useful as a biomarker of this disease.