Complexity Measure For Time Series Research Articles

On the Genuine Relevance of the Data-Driven Signal Decomposition-Based Multiscale Permutation Entropy

Ordinal pattern-based approaches have great potential to capture intrinsic structures of dynamical systems, and therefore, they continue to be developed in various research fields. Among these, the permutation entropy (PE), defined as the Shannon entropy of ordinal probabilities, is an attractive time series complexity measure. Several multiscale variants (MPE) have been proposed in order to bring out hidden structures at different time scales. Multiscaling is achieved by combining linear or nonlinear preprocessing with PE calculation. However, the impact of such a preprocessing on the PE values is not fully characterized. In a previous study, we have theoretically decoupled the contribution of specific signal models to the PE values from that induced by the inner correlations of linear preprocessing filters. A variety of linear filters such as the autoregressive moving average (ARMA), Butterworth, and Chebyshev were tested. The current work is an extension to nonlinear preprocessing and especially to data-driven signal decomposition-based MPE. The empirical mode decomposition, variational mode decomposition, singular spectrum analysis-based decomposition and empirical wavelet transform are considered. We identify possible pitfalls in the interpretation of PE values induced by these nonlinear preprocessing, and hence, we contribute to improving the PE interpretation. The simulated dataset of representative processes such as white Gaussian noise, fractional Gaussian processes, ARMA models and synthetic sEMG signals as well as real-life sEMG signals are tested.

Data-driven fault diagnosis for wind turbines using modified multiscale fluctuation dispersion entropy and cosine pairwise-constrained supervised manifold mapping

Condition monitoring and fault diagnosis of wind turbines is an attractive yet challenging task. This paper presents a novel data-driven fault diagnosis scheme for wind turbines, using refined time-shift multiscale fluctuation-based dispersion entropy (RTSMFDE) and cosine pairwise-constrained supervised manifold mapping (CPCSMM). The developed RTSMFDE can improve multiscale dispersion entropy in complexity measurement of time series, by employing the refined time-shift analysis approach. Moreover, the proposed CPCSMM combines cosine distance metric, supervised theory and sparse global manifold structure, to obtain the most important features, resulting in a low dimensional representation. In this fault diagnosis scheme, a high-dimensional fault feature set is first constructed using RTSMFDE. Then, sensitive features are extracted using CPCSMM. Finally, the low-dimensional feature set is fed to a beetle antennae search-based support vector machine for fault identification. Simulation experiments and wind turbine fault diagnosis experiments were used, to show that the proposed RTSMFDE can comprehensively mine the wind turbine features and outperforms the existing state-of-the-art entropy methods. The visualization results and classification effects of the proposed CPCSMM approach are better than existing unsupervised and supervised dimensionality reduction methods. The proposed data-driven fault diagnosis method can effectively and accurately identify multi-faults and outperforms existing fault diagnosis techniques.

Analysing nystagmus waveforms: a computational framework

We present a new computational approach to analyse nystagmus waveforms. Our framework is designed to fully characterise the state of the nystagmus, aid clinical diagnosis and to quantify the dynamical changes in the oscillations over time. Both linear and nonlinear analyses of time series were used to determine the regularity and complexity of a specific homogenous phenotype of nystagmus. Two-dimensional binocular eye movement recordings were carried out on 5 adult subjects who exhibited a unilateral, uniplanar, vertical nystagmus secondary to a monocular late-onset severe visual loss in the oscillating eye (the Heimann-Bielschowsky Phenomenon). The non-affected eye held a central gaze in both horizontal and vertical planes (± 10 min. of arc). All affected eyes exhibited vertical oscillations, with mean amplitudes and frequencies ranging from 2.0°–4.0° to 0.25–1.5 Hz, respectively. Unstable periodic orbit analysis revealed only 1 subject exhibited a periodic oscillation. The remaining subjects were found to display quasiperiodic (n = 1) and nonperiodic (n = 3) oscillations. Phase space reconstruction allowed attractor identification and the computation of a time series complexity measure—the permutation entropy. The entropy measure was found to be able to distinguish between a periodic oscillation associated with a limit cycle attractor, a quasiperiodic oscillation associated with a torus attractor and nonperiodic oscillations associated with higher-dimensional attractors. Importantly, the permutation entropy was able to rank the oscillations, thereby providing an objective index of nystagmus complexity (range 0.15–0.21) that could not be obtained via unstable periodic orbit analysis or attractor identification alone. These results suggest that our framework provides a comprehensive methodology for characterising nystagmus, aiding differential diagnosis and also permitting investigation of the waveforms over time, thereby facilitating the quantification of future therapeutic managements. In addition, permutation entropy could provide an additional tool for future oculomotor modelling.

Parallel multi-scale entropy and it's application in rolling bearing fault diagnosis

As a prevalent tool for complexity measure of time series, multi-scale entropy has been utilized effectively to extract nonlinear dynamic features from rolling bearing vibration signals. However, the coarse-graining procedure of multi-scale entropy ignores the frequency-domain characteristics of the data, and finally leads to information aliasing and limited state identification capability. In this paper, the frequency characteristic changes of the data in the process of coarse-graining procedure are analyzed, and then a novel nonlinear dynamic method named parallel multi-scale entropy is proposed. In the proposed method, the aliasing effect caused by the reduction of data points is eliminated, and the final estimated sample entropy is replaced by the entropy average of multiple parallel intermediate series. Experimental results show that the proposed parallel multi-scale entropy holds better discrimination and stability compared with the original version, and can provide more excellent feature vectors for intelligent fault diagnosis.

Use of generalized refined composite multiscale fractional dispersion entropy to diagnose the faults of rolling bearing

A typical symptom of vibration signals collected from rolling bearings with local faults is the existence of periodic transients, which makes the intrinsic structure of vibration signals become more and more regular. Generally, the vibration always contains multiple intrinsic oscillatory modes on different scales, which generally is caused by the interaction and coupling of machine components. Therefore, it is necessary to detect the behavior change of complexity of vibration signals in the view of multiple scales for fault information representation. The complexity and nonlinear failure symptom of rolling bearing can be evaluated by the recently proposed nonlinear dynamic tools, such as multiscale entropy (MSE) and its variants. Recently, the improved MSE method, multiscale dispersion entropy (MDE) and its improvement refined composite MDE (RCMDE) are developed to measure the complexity of time domain data. However, the intrinsic shortages of coarse graining approach that used in MDE and RCMDE have limited their application to fault feature representation. In this paper, an improved RCMDE approach named generalized refined composite multiscale fluctuation-based fractional dispersion entropy (GRCMFDE) is proposed to enhance MDE and RCMDE in complexity measurement of time series. GRCMFDE was compared with MPE, MDE, RCMDE by analyzing synthetic simulation signals to verify its advantages. After that, an intelligent fault diagnosis method was proposed by combining GRCMFDE with supervised multi-clustering feature selection and gray wolf optimized SVM for fault classification of rolling bearing. Lastly, the proposed fault diagnostic method was applied to two experimental data set analysis by comparing with multiscale permutation entropy, MDE- and RCMDE-based fault diagnostic methods and the comparison results indicate that the proposed method can effectively diagnose the fault locations and severities of rolling bearing and get a higher fault identifying rate than the comparative methods.

Tangle: A New Measure of Time Series Complexity

Dynamical systems analysis and the study of time series data provide a rich source of information for researchers interested in studying psychological phenomena. While many methods exist for quanti...

Decreased electrocortical temporal complexity distinguishes sleep from wakefulness

In most mammals, the sleep-wake cycle is constituted by three behavioral states: wakefulness (W), non-REM (NREM) sleep, and REM sleep. These states are associated with drastic changes in cognitive capacities, mostly determined by the function of the thalamo-cortical system. The intra-cranial electroencephalogram or electocorticogram (ECoG), is an important tool for measuring the changes in the thalamo-cortical activity during W and sleep. In the present study we analyzed broad-band ECoG recordings of the rat by means of a time-series complexity measure that is easy to implement and robust to noise: the Permutation Entropy (PeEn). We found that PeEn is maximal during W and decreases during sleep. These results bring to light the different thalamo-cortical dynamics emerging during sleep-wake states, which are associated with the well-known spectral changes that occur when passing from W to sleep. Moreover, the PeEn analysis allows us to determine behavioral states independently of the electrodes’ cortical location, which points to an underlying global pattern in the signal that differs among the cycle states that is missed by classical methods. Consequently, our data suggest that PeEn analysis of a single EEG channel could allow for cheap, easy, and efficient sleep monitoring.

Refined time-shift multiscale normalised dispersion entropy and its application to fault diagnosis of rolling bearing

Multiscale entropy (MSE) and multiscale permutation entropy (MPE) are two effective nonlinear dynamic complexity measurement methods of time series that have been applied to many areas for complexity feature extraction in recent years. To overcome the inherent defects of sample entropy and permutation entropy, together with the coarse graining process used in MSE and MPE, based on the recently proposed dispersion entropy (DisEn), the refined time-shift multiscale normalised dispersion entropy (RTSMNDE) is proposed here for the complexity measurement of time series. In the RTSMNDE method, first, to expand DisEn to the multiscale framework, the time-shift multiscale method is used to replace the traditional coarse graining multiscale method. The refining approach is adopted to alleviate the fluctuation of DisEns in larger scale factors, and a normalisation operation is implemented on all entropies to restrain the influence of the parameters on RTSMNDE values. Furthermore, the RTSMNDE is compared with the multiscale dispersion entropy (MDE) by analysing synthetic simulation signals to verify its effectiveness. Based on that, an intelligent fault diagnosis method for rolling bearings is proposed by combining the RTSMNDE for fault feature extraction with the particle swarm optimisation support vector machine for feature classification. Finally, the proposed method is applied to rolling bearing experimental data analysis, and the analysis results show that the proposed method can effectively diagnose the locations and degrees of rolling bearing failures and obtain a higher recognition rate than those of the MPE and MDE-based methods.

Time-Shift Multi-scale Weighted Permutation Entropy and GWO-SVM Based Fault Diagnosis Approach for Rolling Bearing.

Multi-scale permutation entropy (MPE) is an effective nonlinear dynamic approach for complexity measurement of time series and it has been widely applied to fault feature representation of rolling bearing. However, the coarse-grained time series in MPE becomes shorter and shorter with the increase of the scale factor, which causes an imprecise estimation of permutation entropy. In addition, the different amplitudes of the same patterns are not considered by the permutation entropy used in MPE. To solve these issues, the time-shift multi-scale weighted permutation entropy (TSMWPE) approach is proposed in this paper. The inadequate process of coarse-grained time series in MPE was optimized by using a time shift time series and the process of probability calculation that cannot fully consider the symbol mode is solved by introducing a weighting operation. The parameter selections of TSMWPE were studied by analyzing two different noise signals. The stability and robustness were also studied by comparing TSMWPE with TSMPE and MPE. Based on the advantages of TSMWPE, an intelligent fault diagnosis method for rolling bearing is proposed by combining it with gray wolf optimized support vector machine for fault classification. The proposed fault diagnostic method was applied to two cases of experimental data analysis of rolling bearing and the results show that it can diagnose the fault category and severity of rolling bearing accurately and the corresponding recognition rate is higher than the rate provided by the existing comparison methods.

Complexity mapping of a photonic integrated circuit laser using a correlation-dimension-based approach

Quantifying complexity from experimental time series generated by nonlinear systems, including laser systems, remains a challenge. Methods that are based on entropy, such as permutation entropy (PE), have proven to be useful tools for the relative measure of time series complexity. However, the numerical value of PE is not readily linked to a specific type of dynamical output. Thus, the quest to calculate quantitatively meaningful fractal dimension values, such as the correlation dimension (CD), from experimental signals, is still important. A protocol for calculating minimum gradient values and their spread, an integral part of CD analysis, is used here. Minimum gradient values with small spread are presented as approximate CD values. Here-in we report mapping these values, derived from analyzing experimental time series, obtained from a 4-section photonic integrated circuit laser (PICL) across a large parameter space. The PICL is an integrated form of a semiconductor laser subject to controllable optical feedback system. The minimum gradient/approximate CD mapping shows it has some qualitatively different map regions in its dynamics as compared to a free-space-based equivalent system. We show that the minimum gradient values give insight into the dynamics even when approximate CD values cannot be determined. The agreement between the qualitative features of permutation entropy mapping and minimum gradient/approximate CD value mapping provides further support for this. Regions of time series with close to periodic and quasi-periodic dynamics are identifiable using minimum gradient value maps.

Embedded Dimension and Time Series Length. Practical Influence on Permutation Entropy and Its Applications.

Permutation Entropy (PE) is a time series complexity measure commonly used in a variety of contexts, with medicine being the prime example. In its general form, it requires three input parameters for its calculation: time series length N, embedded dimension m, and embedded delay . Inappropriate choices of these parameters may potentially lead to incorrect interpretations. However, there are no specific guidelines for an optimal selection of N, m, or , only general recommendations such as , , or . This paper deals specifically with the study of the practical implications of , since long time series are often not available, or non-stationary, and other preliminary results suggest that low N values do not necessarily invalidate PE usefulness. Our study analyses the PE variation as a function of the series length N and embedded dimension m in the context of a diverse experimental set, both synthetic (random, spikes, or logistic model time series) and real–world (climatology, seismic, financial, or biomedical time series), and the classification performance achieved with varying N and m. The results seem to indicate that shorter lengths than those suggested by are sufficient for a stable PE calculation, and even very short time series can be robustly classified based on PE measurements before the stability point is reached. This may be due to the fact that there are forbidden patterns in chaotic time series, not all the patterns are equally informative, and differences among classes are already apparent at very short lengths.

Refined Composite Multivariate Multiscale Dispersion Entropy and Its Application to Fault Diagnosis of Rolling Bearing

Many nonlinear dynamic and statistic methods, including multiscale sample entropy (MSE) and multiscale fuzzy entropy (MFE), have been widely studied and employed to fault diagnosis of the rolling bearing. Multiscale dispersion entropy (MDE) is a powerful tool for complexity measure of time series, and compared with MSE and MFE, it gets much better performance and costs less time for computation. Since single-channel time series analysis will cause information missing, inspired by multivariate multiscale sample entropy (MMSE) and multivariate multiscale fuzzy entropy (MMFE), refined composite multivariate multiscale dispersion entropy (RCMMDE) was proposed in this paper. After that, RCMMDE was compared with MDE, MMSE, and MMFE by analyzing synthetic signals and the results show that the RCMMDE has certain advantages in terms of robustness. A hybrid fault diagnostics approach is proposed for rolling bearing with a combination of RCMMDE, multi-cluster feature selection, and support vector machine. Also, the proposed method is compared with MDE, MMSE, and MMFE, as well as multivariate multiscale dispersion entropy-based fault diagnosis methods by analyzing the experimental data of rolling bearing, and the result shows that the proposed method gets a higher identification rate than the existing other fault diagnosis methods.

Recent Improvements on Complexity Measures for Time Series

Recent Improvements on Complexity Measures for Time Series

Sigmoid-based refined composite multiscale fuzzy entropy and t-SNE based fault diagnosis approach for rolling bearing

Multiscale fuzzy entropy (MFE) has been taken as a prevalent tool for complexity measure of time series. However, the entropy estimation of MFE is sensitive to the preset parameters and will cause an inaccurate and even undefined entropy when the data is not long enough. In this paper the Sigmoid-based refined composite multiscale fuzzy entropy (SRCMFE) is proposed to improve the performance of MFE for complexity measure of short time series. The proposed SRCMFE method is compared with MFE and Sigmoid-based multiscale fuzzy entropy by analyzing Gaussian white noise and 1/f noise to verify its effectiveness and the analysis result indicate that SRCMFE method holds a better distinguish capacity and robustness and can reflect more complexity information of time series. Then SRCMFE is used to the dynamical complexity analysis of mechanical vibration signals and based on that a new fault diagnosis approach for rolling bearing is put forward by combining SRCMFE with t-distributed stochastic neighbor embedding (t-SNE) for feature dimension and the recently proposed variable predictive models based class discrimination (VPMCD) method for fault pattern recognition. In the proposed method, firstly, SRCMFE is utilized to extract the complexity characteristic related with fault information from vibration signals of rolling bearing. Then the feature dimension reduction method named t-SNE is adopt to obtain a low dimensional manifold features of rolling bearing. Next, the VPMCD method is employed for multi-fault classifier construction to fulfill an intelligent fault diagnosis according to the intrinsic inner relationships hidden in the fault features. Finally, the proposed fault diagnosis approach is applied to the experimental data analysis of rolling bearing and is compared with the same kinds of fault diagnosis methods through experiment data analysis. The analysis and comparison results indicate that the proposed method is very effective in distinguishing the fault categories of rolling bearings and can get a higher identifying rate than the contrast methods.

Multiscale Distribution Entropy and t-Distributed Stochastic Neighbor Embedding-Based Fault Diagnosis of Rolling Bearings

As a nonlinear dynamic method for complexity measurement of time series, multiscale entropy (MSE) has been successfully applied to fault diagnosis of rolling bearings. However, the MSE algorithm is sensitive to the predetermined parameters and depends heavily on the length of the time series and MSE may yield an inaccurate estimation of entropy or undefined entropy when the length of time series is too short. To improve the robustness of complexity measurement for short time series, a novel nonlinear parameter named multiscale distribution entropy (MDE) was proposed and employed to extract the nonlinear complexity features from vibration signals of rolling bearing in this paper. Combining with t-distributed stochastic neighbor embedding (t-SNE) for feature dimension reduction and Kriging-variable predictive models based class discrimination (KVPMCD) for automatic identification, a new intelligent fault diagnosis method for rolling bearings was proposed. Finally, the proposed approach was applied to analyze the experimental data of rolling bearings and the results indicated that the proposed method could distinguish the different fault categories of rolling bearings effectively.

Decreased complexity of glucose dynamics preceding the onset of diabetes in mice and rats.

Continuous glucose monitoring (CGM) is a platform to measure blood glucose (BG) levels continuously in real time with high enough resolution to document their underlying fluctuations. Multiscale entropy (MSE) analysis has been proposed as a measure of time-series complexity, and when applied to clinical CGM data, MSE analysis revealed that diabetic patients have lower MSE complexity in their BG time series than healthy subjects. To determine if the clinical observations on complexity of glucose dynamics can be back-translated to relevant preclinical species used routinely in diabetes drug discovery, we performed CGM in both mouse (ob/ob) and rat (Zucker Diabetic Fatty, ZDF) models of diabetes. We demonstrate that similar to human data, the complexity of glucose dynamics is also decreased in diabetic mice and rats. We show that low complexity of glucose dynamics is not simply a reflection of high glucose values, but rather reflective of the underlying disease state (i.e. diabetes). Finally, we demonstrate for the first time that the complexity of glucose fluctuations in ZDF rats, as probed by MSE analysis, is decreased prior to the onset of overt diabetes, although complexity undergoes further decline during the transition to frank diabetes. Our study suggests that MSE could serve as a novel biomarker for the progression to diabetes and that complexity studies in preclinical models could offer a new paradigm for early differentiation, and thereby, selection of appropriate clinical candidate molecules to be tested in human clinical trials.

Characterizing time series via complexity-entropy curves.

The search for patterns in time series is a very common task when dealing with complex systems. This is usually accomplished by employing a complexity measure such as entropies and fractal dimensions. However, such measures usually only capture a single aspect of the system dynamics. Here, we propose a family of complexity measures for time series based on a generalization of the complexity-entropy causality plane. By replacing the Shannon entropy by a monoparametric entropy (Tsallis q entropy) and after considering the proper generalization of the statistical complexity (q complexity), we build up a parametric curve (the q-complexity-entropy curve) that is used for characterizing and classifying time series. Based on simple exact results and numerical simulations of stochastic processes, we show that these curves can distinguish among different long-range, short-range, and oscillating correlated behaviors. Also, we verify that simulated chaotic and stochastic time series can be distinguished based on whether these curves are open or closed. We further test this technique in experimental scenarios related to chaotic laser intensity, stock price, sunspot, and geomagnetic dynamics, confirming its usefulness. Finally, we prove that these curves enhance the automatic classification of time series with long-range correlations and interbeat intervals of healthy subjects and patients with heart disease.

Binary and multi-class motor imagery using Renyi entropy for feature extraction

Entropy, the complexity measures for time series, has found numerous successful applications in brain signal analysis such as detection of epileptic seizure and monitoring the depth of anesthesia. Renyi entropy generalizes the well-known Shannon entropy, and hence providing better flexibility in application to real data. The objective of this paper is to evaluate the effectiveness of Renyi entropy as feature extraction method for motor imagery (MI)-based brain–computer interface (BCI). In this paper, Renyi entropy has been implemented in MI systems of various settings using BCI competition data sets. The classification accuracy of Renyi entropy in all data sets is benchmarked against common spatial pattern (CSP), the state-of-the-art feature extraction method. For common binary class data sets, Renyi entropy achieves an accuracy of approximately 3.4 % higher for BCI Competition II Data Set III and 0.87 % lower for BCI Competition III Data Set I, and there is no difference in accuracy for BCI Competition III Data Set IVc when compared against conventional CSP. In small sample setting, the average classification accuracy using Renyi entropy is approximately 3.4 and 0.3 % higher as compared to conventional CSP and best performing variant of regularized CSP, respectively. In addition, Renyi entropy is also compared to other chaos-inspired feature extraction methods, namely Katz and Higuchi which are implemented for MI systems by earlier researchers. The effect of implementing Renyi entropy on multiple narrower frequency sub-bands is also investigated in this study. In multi-class setting, Renyi entropy shows no statistical difference (p = 0.6022, paired t test) in accuracy when compared to the algorithm of the BCI competition winner. However, Renyi entropy has an advantage of requiring only one single application of feature extraction regardless of the number of classes, while multiple implementation of CSP is required as CSP is originally designed for binary class problem. The successful application of Renyi entropy in all the aforementioned settings indicates that Renyi entropy is a viable feature extraction alternative for MI-based BCI systems.

A permutation Lempel-Ziv complexity measure for EEG analysis

ObjectiveIn this study we develop a new complexity measure of time series by combining ordinal patterns and Lempel-Ziv complexity (LZC) for quantifying the dynamical changes of EEG. MethodsA neural mass model (NMM) was used to simulate EEG data and test the performance of the permutation Lempel-Ziv complexity (PLZC) in tracking the dynamical changes of signals against different white noise levels. Then, the PLZC was applied to real EEG data to investigate whether it was able to detect the different states of anesthesia and epileptic seizures. The Z-score model, two-way ANOVA and t-test were used to estimate the significance of the results. ResultsPLZC could successfully track the dynamical changes of EEG series generated by the NMM. Compared with the other four classical LZC based methods, the PLZC was most robust against white noise. In real data analysis, PLZC was effective in differentiating the different anesthesia states and sensitive in detecting epileptic seizures. ConclusionsPLZC is simple, robust and effective for quantifying the dynamical changes of EEG. SignificanceWe suggest that PLZC is a potential nonlinear method for characterizing the changes in EEG signal.

Electromyographic Permutation Entropy Quantifies Diaphragmatic Denervation and Reinnervation

Spontaneous reinnervation after diaphragmatic paralysis due to trauma, surgery, tumors and spinal cord injuries is frequently observed. A possible explanation could be collateral reinnervation, since the diaphragm is commonly double-innervated by the (accessory) phrenic nerve. Permutation entropy (PeEn), a complexity measure for time series, may reflect a functional state of neuromuscular transmission by quantifying the complexity of interactions across neural and muscular networks. In an established rat model, electromyographic signals of the diaphragm after phrenicotomy were analyzed using PeEn quantifying denervation and reinnervation. Thirty-three anesthetized rats were unilaterally phrenicotomized. After 1, 3, 9, 27 and 81 days, diaphragmatic electromyographic PeEn was analyzed in vivo from sternal, mid-costal and crural areas of both hemidiaphragms. After euthanasia of the animals, both hemidiaphragms were dissected for fiber type evaluation. The electromyographic incidence of an accessory phrenic nerve was 76%. At day 1 after phrenicotomy, PeEn (normalized values) was significantly diminished in the sternal (median: 0.69; interquartile range: 0.66–0.75) and mid-costal area (0.68; 0.66–0.72) compared to the non-denervated side (0.84; 0.78–0.90) at threshold p<0.05. In the crural area, innervated by the accessory phrenic nerve, PeEn remained unchanged (0.79; 0.72–0.86). During reinnervation over 81 days, PeEn normalized in the mid-costal area (0.84; 0.77–0.86), whereas it remained reduced in the sternal area (0.77; 0.70–0.81). Fiber type grouping, a histological sign for reinnervation, was found in the mid-costal area in 20% after 27 days and in 80% after 81 days. Collateral reinnervation can restore diaphragm activity after phrenicotomy. Electromyographic PeEn represents a new, distinctive assessment characterizing intramuscular function following denervation and reinnervation.