Permutation Entropy Algorithm Research Articles

Forecasting Gate-Front Water Levels Using a Coupled GRU–TCN–Transformer Model and Permutation Entropy Algorithm

Water level forecasting has significant impacts on transportation, agriculture, and flood control measures. Accurate water level values can enhance the safety and efficiency of water conservancy hub operation scheduling, reduce flood risks, and are essential for ensuring sustainable regional development. Addressing the nonlinearity and non-stationarity characteristics of gate-front water level sequences, this paper introduces a gate-front water level forecasting method based on a GRU–TCN–Transformer coupled model and permutation entropy (PE) algorithm. Firstly, an analysis method combining Singular Spectrum Analysis (SSA) and Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) is used to separate the original water level data into different frequency modal components. The PE algorithm subsequently divides each modal component into sequences of high and low frequencies. The GRU model is applied to predict the high-frequency sequence part, while the TCN–Transformer combination model is used for the low-frequency sequence part. The forecasting from both models are combined to obtain the final water level forecasting value. Multiple evaluation metrics are used to assess the forecasting performance. The findings indicate that the combined GRU–TCN–Transformer model achieves a Mean Absolute Error (MAE) of 0.0154, a Root Mean Square Error (RMSE) of 0.0205, and a Coefficient of Determination (R2) of 0.8076. These metrics indicate that the model outperforms machine learning Support Vector Machine (SVM) models, GRU models, Transformer models, and TCN–Transformer combination models in forecasting performance. The forecasting results have high credibility. This model provides a new reference for improving the accuracy of gate-front water level forecasting and offers significant insights for water resource management and flood prevention, demonstrating promising application prospects.

CEEMDAN combined wavelet denoising improved the MW cycle slip detection algorithm

In the preprocessing of high-precision navigation and positioning data, the most widely used MW combination cycle slip detection method is greatly affected by pseudorange noise. It has issues such as missing small cycle slips and failing to promptly reset the recursive averaging process after cycle slip detection failure, which leads to subsequent threshold divergence. This paper proposes an improved MW combination cycle slip detection method based on Complete Ensemble Empirical Mode Decomposition (CEEMDAN), permutation entropy, and wavelet denoising, which uses CEEMDAN to decompose the cycle slip signal into a series of intrinsic modal functions (IMFs) and then selects the IMFs that require denoising through the permutation entropy algorithm, and the wavelet denoising technique is combined to eliminate the residual noise further, so that the noise can be removed more accurately. Experimental results show that compared with the original MW algorithm, the proposed improved method can effectively reduce the influence of pseudo-range noise, and reduce the false detection rate of cycle slip from 1.6% and 6–0%. All small period slips can be successfully detected in complex noise environments, avoiding the missed detection of the original MW algorithm and the related threshold divergence problems.

Application of cross-channel multiscale permutation entropy in measuring multichannel data complexity.

Entropy is a pivotal concept in nonlinear dynamics, revealing chaos, self-organization, and information transmission in complex systems. Permutation entropy, due to its computational efficiency and lower data length requirements, has found widespread use in various fields. However, in the age of multi-channel data, existing permutation entropy methods are limited in capturing cross-channel information. This paper presents cross-channel multiscale permutation entropy algorithm, and the proposed algorithm can effectively capture the cross-channel information of multi-channel dataset. The major modification lies in the concurrent frequency counting of specific events during the calculation steps. The algorithm improves phase space reconstruction and mapping, enhancing the capability of multi-channel permutation entropy methods to extract cross-channel information. Simulation and real-world multi-channel data analysis demonstrate the superiority of the proposed algorithm in distinguishing different types of data. The improvement is not limited to one specific algorithm and can be applied to various multi-channel permutation entropy variants, making them more effective in uncovering information across different channels.

Tsallis Entropy-Based Complexity-IPE Casualty Plane: A Novel Method for Complex Time Series Analysis.

Due to its capacity to unveil the dynamic characteristics of time series data, entropy has attracted growing interest. However, traditional entropy feature extraction methods, such as permutation entropy, fall short in concurrently considering both the absolute amplitude information of signals and the temporal correlation between sample points. Consequently, this limitation leads to inadequate differentiation among different time series and susceptibility to noise interference. In order to augment the discriminative power and noise robustness of entropy features in time series analysis, this paper introduces a novel method called Tsallis entropy-based complexity-improved permutation entropy casualty plane (TC-IPE-CP). TC-IPE-CP adopts a novel symbolization approach that preserves both absolute amplitude information and inter-point correlations within sequences, thereby enhancing feature separability and noise resilience. Additionally, by incorporating Tsallis entropy and weighting the probability distribution with parameter q, it integrates with statistical complexity to establish a feature plane of complexity and entropy, further enriching signal features. Through the integration of multiscale algorithms, a multiscale Tsallis-improved permutation entropy algorithm is also developed. The simulation results indicate that TC-IPE-CP requires a small amount of data, exhibits strong noise resistance, and possesses high separability for signals. When applied to the analysis of heart rate signals, fault diagnosis, and underwater acoustic signal recognition, experimental findings demonstrate that TC-IPE-CP can accurately differentiate between electrocardiographic signals of elderly and young subjects, achieve precise bearing fault diagnosis, and identify four types of underwater targets. Particularly in underwater acoustic signal recognition experiments, TC-IPE-CP achieves a recognition rate of 96.67%, surpassing the well-known multi-scale dispersion entropy and multi-scale permutation entropy by 7.34% and 19.17%, respectively. This suggests that TC-IPE-CP is highly suitable for the analysis of complex time series.

A novel approach to precipitation prediction using a coupled CEEMDAN-GRU-Transformer model with permutation entropy algorithm.

The accurate forecasting of precipitation in the upper reaches of the Yellow River is imperative for enhancing water resources in both the local and broader Yellow River basin in the present and future. While many models exist for predicting precipitation by analyzing historical data, few consider the impact of different frequency sequences on model accuracy. In this study, we propose a coupled monthly precipitation prediction model that leverages the adaptive noise complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), gated recurrent unit neural network (GRU), and attention mechanism-based transformer model. The permutation entropy (PE) algorithm is employed to partition the data processed by CEEMDAN into different frequencies, with different models utilized to predict different frequencies. The predicted results are subsequently combined to obtain the monthly precipitation prediction value. The model is applied to precipitation prediction in four regions in the upper reaches of the Yellow River and compared with other models. Evaluation results demonstrate that the CEEMDAN-GRU-Transformer model outperforms other models in predicting precipitation for these regions, with a coefficient of determination R2 greater than 0.8. These findings suggest that the proposed model provides a novel and effective method for improving the accuracy of regional medium and long-term precipitation prediction.

A VMD-PE-SG denoising method based on K–L divergence for satellite atomic clock

The time scale of the global navigation satellite system (GNSS) is the core element for its position, navigation and timing services. A highly stable atomic clock is essential to ensure the reliability of the GNSS time scale. This study proposed a novel hybrid denoising model combining variational mode decomposition (VMD), K–L divergence, permutation entropy (PE), and Savitzky–Golay (SG) filter for satellite atomic clocks. Firstly, the key parameter of VMD is solved efficiently by taking the minimum sum of K–L divergence of decomposed modes as the constraint condition, and the optimised parameters are applied to the decomposition process. On this basis, the PE algorithm is used to determine the modes decomposed by VMD into signal-dominant and noise-dominant components by searching for the mutation of PE value at two adjacent points. Finally, the noise-dominant components are denoised by the SG filter and then reconstructed with the signal-dominant components to form the denoised signal. The analysis of the simulated signal shows that the method can effectively remove noise from the simulated signal, and the resulting denoised signal is similar to the pure signal. Compared with commonly used ensemble empirical mode decomposition and wavelet denoising methods, the signal-noise ratio of the proposed method is improved by 21.2% and 28.9%, and the root mean square error is improved by 24.1% and 29.8%, respectively. The results of experimental data testify that the K–L VMD-PE-SG-based denoising method can significantly reduce the dominant noise within one day, thus effectively improving the short to medium-term frequency stability. Compared with the original signal, the stability of the smoothing time within 76 800 s is generally improved, and the degree of improvement depends on the type of atomic clock and the smoothing time.

Research on Low-Resistance Grounding Fault Line Selection Based on VMD with PE and K-Means Clustering Algorithm

Aiming at the fault line selection problem in the single-phase grounding system of the distribution network, a new fault line selection method based on VMD and permutation entropy feature extraction combined with K-means clustering algorithm is proposed. This method is a hybrid algorithm that can effectively identify fault line selection. Firstly, a simulation model is built and its zero sequence current is collected. The variational modal decomposition method is used to decompose the collected zero-sequence current into multiple intrinsic modal functions, which can not only effectively reduce the influence of harmonic components and noise in the characteristic signal but also facilitate the calculation. The extracted intrinsic mode function is calculated by permutation entropy (PE), and the calculated entropy value is constructed into a matrix to highlight the fault characteristics of the line; then, the matrix is subjected to K-means cluster analysis through the preprocessing algorithm and the faulty line is correctly distinguished. Then, regression verification is performed. Finally, it is verified by the recorded wave data of the real test site and then analyzed and compared with other algorithms. The proposed method shows that when a single-phase ground fault occurs, the ground fault line selection can be effectively identified under different transition resistances, grounding resistances, and fault distances. Therefore, this method can accurately identify the fault line selection, and the accuracy rate is 100%, which has a certain use value.

Application of permutation entropy and factorization network in the classification of congenital heart disease

AbstractOwing to the nonstationary feature of heart sound signal which is often disturbed by noise, a method based on permutation entropy and factorization network in the classification of heart sound of congenital heart disease is proposed. Firstly, the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) of normal and abnormal heart sound signals is performed to obtain several intrinsic mode function (IMF) components. Secondly, the optimal parameters in the permutation entropy algorithm are determined by C_C algorithm, and then according to the algorithm, the entropy values of the IMF components of each order are calculated to form high‐dimensional eigenvectors. Finally, the recommended models of factorization machines (FMs) are used to classify the heart sounds and the generalisation performance of the FM network model are evaluated from accuracy and Area Under ROC Curve (AUC). The results show that the FM accuracy is 0.865, the AUC value is 0.887; the SVM accuracy is 0.812, and the AUC value is 0.805.

Permutation entropy-based improved uniform phase empirical mode decomposition for mechanical fault diagnosis

Uniform phase empirical mode decomposition (UPEMD) is an effective signal separation method, which is proposed to solve the mode mixing phenomenon of empirical mode decomposition (EMD) by adding uniform phase sinusoidal signal as masking signal. However, the decomposition effect of UPEMD depends on the selection of amplitude and phase number of masking signal. Besides, there are also some issues such as noise residue and incomplete decomposition that need to be solved. In this paper, a novel permutation entropy-based improved uniform phase empirical mode decomposition (PEUPEMD) method is proposed to address these problems. In PEUPEMD method, first of all, the sinusoidal signals with uniform phases are added to the raw signal as masking signals. Second, the superimposed signal is decomposed using EMD and the final intrinsic mode functions are obtained via an ensemble way. Third, the obtained IMFs with high-frequency are detected by permutation entropy algorithm. Finally, the residual signal containing low-frequency components is decomposed through EMD completely. Last, the simulated signal and tested data are applied to verify the feasibility of PEUPEMD via comparing it with EMD, UPEMD, CEEMDAN and ICEEMDAN methods. The results analysis indicated that PEUPEMD was superior to the comparative methods in decomposing accuracy and mode mixing suppression.

A Modified Multivariable Complexity Measure Algorithm and Its Application for Identifying Mental Arithmetic Task.

Properly measuring the complexity of time series is an important issue. The permutation entropy (PE) is a widely used as an effective complexity measurement algorithm, but it is not suitable for the complexity description of multi-dimensional data. In this paper, in order to better measure the complexity of multi-dimensional time series, we proposed a modified multivariable PE (MMPE) algorithm with principal component analysis (PCA) dimensionality reduction, which is a new multi-dimensional time series complexity measurement algorithm. The analysis results of different chaotic systems verify that MMPE is effective. Moreover, we applied it to the comlexity analysis of EEG data. It shows that the person during mental arithmetic task has higher complexity comparing with the state before mental arithmetic task. In addition, we also discussed the necessity of the PCA dimensionality reduction.

Dynamic efficiency of China’s commodity futures market through the lens of high frequency data

In this paper, we use the permutation entropy algorithm to derive the static and dynamic permutation entropy of commodity futures, and to evaluate the effectiveness of main products in China’s commodity futures market. The intraday data of six varieties belonging to six categories in China’s commodity futures market are taken as samples. We find the following: (1) The return distribution of the main varieties shows high peaks, fat tails and asymmetry, and follows the biased random walk distribution characteristics; (2) The permutation entropy of all varieties decreases significantly in the same time window, during which the price volatility of major commodity markets rises. And the time window coincides with the impact time of COVID-19 epidemic; (3) By comparing the distribution of permutation entropy of main varieties in different stages of event shock, we found that the mean value of permutation entropy decreases significantly during the process of event shock, and the price fluctuates greatly. Therefore, the significant decrease of permutation entropy is a valuable warning signal for regulators and investors.

A New Joint Denoising Algorithm for High‐G Calibration of MEMS Accelerometer Based on VMD‐PE‐Wavelet Threshold

Recently, the High‐G MEMS accelerometer (HGMA) has been used in navigation, mechanical property detection, consumer electronics, and other fields widely. As the core component of a measuring system, it is very crucial to enhance the calibration accuracy of the accelerometer. In order to remove the noises in the accelerometer output signals to enhance its calibration accuracy, a combined denoising method which combines variational mode decomposition (VMD) with permutation entropy (PE) and wavelet threshold is given in this article. For the sake of overcoming the defect of signal distortion caused by the traditional denoising methods, this joint denoising method combines the good decomposition characteristics of VMD and the good denoising ability of wavelet threshold and introduces PE as a judgment criterion to achieve a good balance between denoising effect and signal fidelity. The combination of PE and VMD not only avoids the phenomenon of mode aliasing but also improves the ability to identify the noise components, which makes the wavelet threshold denoising more specific. Firstly, some intrinsic mode functions (IMFs) are obtained by using VMD to decompose the complex signal containing noise which is outputted from the accelerometer. Secondly, the IMF components can be divided into noise IMF components, mixed IMF components, and useful IMF components by PE algorithm. Thirdly, the noise IMF components can be discarded directly, and then the mixed IMF components can be denoised by wavelet threshold to obtain the noiseless IMF components; in addition, the useful IMF components need to be retained. Finally, the final denoising signal can be obtained by reconstructing the IMF components which have been denoised by the wavelet threshold and the useful IMF components retained before denoising. The experimental results prove that the combined denoising algorithm combines the merits of VMD, PE, and wavelet threshold, and this new algorithm has a good performance in the calibration denoising of accelerometer. Compared with the serious signal distortion caused by using only EMD or wavelet threshold, this method not only has a good denoising effect (the noises in the static part are eliminated by 99.97% and the SNR of the dynamic part is raised to 18.56) but also can maintain a good signal fidelity (the error of shock peak amplitude is 3.4%, the error of vibration peak amplitude is 0.4%, and the correlation coefficient between the denoising signals and dynamic part is as high as 0.982).

Integrating ultra weak luminescence properties and multi-scale permutation entropy algorithm to analyze freshness degree of wheat kernel

With the improvement of agricultural productivity, the wheat yield in China is soaring, which extremely burdens the quality guarantee pressure both in circulation and storage. According to the incomplete statistics, we loss millions of tons wheat every year because of freshness degree, mould, insects intrusion and so on. Thus finding an effective method to detect wheat kernel’s freshness degree is of necessity for guiding scientific storage of wheat. Up to now, process of wheat kernel’s freshness degree analysis can be divided into sensory evaluation, biochemical detection and physical detection method. Due to the inevitable deficits existed among these traditional methods, a novel method to detect freshness degree of wheat kernel based on ultra weak luminescence(UWL) characteristics integrated with multi-scale permutation entropy(MPE) algorithm has been proposed. Through calculating the mean value, variance, standard deviation, and MPE of four year’s wheat kernel separately, then further resorting to back propagation neural network to make the classification. Compared with traditional analogy methods, the new method not only improves the correct rate from 82.5%–92.5%, but also saves running time nearly three times shorter than PE algorithm. Shown by the experiment results, the new method proposed above is an efficient and feasible way to analyze freshness degree of wheat kernel.

Slope Entropy: A New Time Series Complexity Estimator Based on Both Symbolic Patterns and Amplitude Information

The development of new measures and algorithms to quantify the entropy or related concepts of a data series is a continuous effort that has brought many innovations in this regard in recent years. The ultimate goal is usually to find new methods with a higher discriminating power, more efficient, more robust to noise and artifacts, less dependent on parameters or configurations, or any other possibly desirable feature. Among all these methods, Permutation Entropy (PE) is a complexity estimator for a time series that stands out due to its many strengths, with very few weaknesses. One of these weaknesses is the PE’s disregarding of time series amplitude information. Some PE algorithm modifications have been proposed in order to introduce such information into the calculations. We propose in this paper a new method, Slope Entropy (SlopEn), that also addresses this flaw but in a different way, keeping the symbolic representation of subsequences using a novel encoding method based on the slope generated by two consecutive data samples. By means of a thorough and extensive set of comparative experiments with PE and Sample Entropy (SampEn), we demonstrate that SlopEn is a very promising method with clearly a better time series classification performance than those previous methods.

A Rolling Bearing Fault Diagnosis Method Based on EMD and Quantile Permutation Entropy

The vibration signals resulting from rolling bearings are nonlinear and nonstationary, and an approach for the fault diagnosis of rolling bearings using the quantile permutation entropy and EMD (empirical mode decomposition) is proposed. Firstly, the EMD is used to decompose the rolling bearings vibration signal, and several IMFs (intrinsic mode functions) spanning different scales are obtained. Secondly, aiming at the shortcomings of the permutation entropy algorithm, a new permutation entropy algorithm based on sample quantile is proposed, and the quantile permutation entropy of the first few IMFs, which contain the main fault information, is calculated. The quantile permutation entropies are accordingly seen as the characteristic vector and then input to the particle swarm optimization and support vector machine. Finally, the proposed method is applied to the experimental data. The analysis results show that the proposed approach can effectively achieve fault diagnosis of rolling bearings.

Design of a Network Permutation Entropy and Its Applications for Chaotic Time Series and EEG Signals

Measuring the complexity of time series provides an important indicator for characteristic analysis of nonlinear systems. The permutation entropy (PE) is widely used, but it still needs to be modified. In this paper, the PE algorithm is improved by introducing the concept of the network, and the network PE (NPE) is proposed. The connections are established based on both the patterns and weights of the reconstructed vectors. The complexity of different chaotic systems is analyzed. As with the PE algorithm, the NPE algorithm-based analysis results are also reliable for chaotic systems. Finally, the NPE is applied to estimate the complexity of EEG signals of normal healthy persons and epileptic patients. It is shown that the normal healthy persons have the largest NPE values, while the EEG signals of epileptic patients are lower during both seizure-free intervals and seizure activity. Hence, NPE could be used as an alternative to PE for the nonlinear characteristics of chaotic systems and EEG signal-based physiological and biomedical analysis.

Generalized phase permutation entropy algorithm based on two-index entropy

Generalized permutation entropy $({\rm~PE}_{q,\delta})$ with appropriate parameters can amplify minor changes in a system; however, the phase of the signal contains more critical information than the amplitude. This paper introduces the phase information into the generalized permutation entropy and proposes the generalized phase permutation entropy $({\rm~PPE}_{q,\delta})$ algorithm. Moreover, we verify the advantages of ${\rm~PPE}_{q,\delta}$ in detecting the dynamic changes in the system, analyze the influence of $q$, $\delta$ selection on the dynamic change in the system, and explore the effect of data length and noise for ${\rm~PPE}_{q,\delta}$. Finally, the ${\rm~PPE}_{q,\delta}$ is applied to analyze abnormal ECG signals. When the values of $q$, $\delta$ are the same, the ${\rm~PPE}_{q,\delta}$ has a more significant effect on the detection of the same dynamic change. Whether for use in a logistic map or in detecting abnormal ECG signal dynamic changes, when $q>0$ and $\delta>0$, the effect of ${\rm~PE}_{q,\delta}$ and ${\rm~PPE}_{q,\delta}$ can be improved by decreasing $q$ value, increasing $\delta$ value, or simultaneously changing both values. Furthermore, the change in data length has no effect for ${\rm~PE}_{q,\delta}$ and ${\rm~PPE}_{q,\delta}$, and both are robust to noise.

Monitoring of early looseness of multi-bolt connection: a new entropy-based active sensing method without saturation

Monitoring the looseness of bolted connection, especially at the inception or early stage, is important to ensure its integrity and the wellbeing of structures. In the past decades, significant progress, including several state-of-the-art structural health monitoring approaches, has been made to characterize health status of bolted connections. One of these approaches is the active sensing method with merits such as low cost and easy-to-operate; however, its current damage index (DI) may saturate at high levels of bolt preload, which will impede its application in detecting early looseness of bolted connections. As the main contribution of this paper, we apply the multiscale permutation entropy algorithm to develop a new entropy-based DI that eliminates the saturation phenomenon of current DI (i.e. the signal energy), enhancing the performance of the piezoceramic-enabled active sensing method in monitoring bolt early looseness. Multiple experiments were conducted on a multi-bolt connection with two surface-bonded piezoceramic transducers to verify the effectiveness of the proposed method. Finally, after processing the received signals through the new DI, we demonstrated that the proposed method effectively eliminated the saturation problem, which is significant in detecting early looseness of bolt connections.

A Novel Denoising Algorithm of Electromagnetic Ultrasonic Detection Signal Based on Improved EEMD Method

In this paper, we propose a new denoising algorithm for electromagnetic ultrasonic signals based on the improved EEMD method, which can adaptively adjust for added noise and average times in different noisy environments, so that the effect of the residual difference of white noise on the results can be eliminated as far as possible. First, the way to add white noise in the EEMD method is processed, and then the permutation entropy algorithm is used to identify the nature of the components obtained during the decomposition. Then the wavelet transform modulus maximum denoising method is used to deal with the IMF components of the high-frequency part obtained before. Finally, the processed IMF results and residual difference are summed up. The results show that after processing, the noise component in the signal is less and the original information is more reserved, which prevents the signal distortion to a great extent and provides more effective data for subsequent processing. In the experiment, the crack defect data collected by the electromagnetic ultrasonic experiment system were processed by the improved EEMD method. Compared with the traditional EEMD method, it can retain the information of crack location more accurately, which proves the effectiveness of the proposed method.

Classification of glucose records from patients at diabetes risk using a combined permutation entropy algorithm

Background and objectives: The adoption in clinical practice of electronic portable blood or interstitial glucose monitors has enabled the collection, storage, and sharing of massive amounts of glucose level readings. This availability of data opened the door to the application of a multitude of mathematical methods to extract clinical information not discernible with conventional visual inspection. The objective of this study is to assess the capability of Permutation Entropy (PE) to find differences between glucose records of healthy and potentially diabetic subjects.Methods: PE is a mathematical method based on the relative frequency analysis of ordinal patterns in time series that has gained a lot of attention in the last years due to its simplicity, robustness, and performance. We study in this paper the applicability of this method to glucose records of subjects at risk of diabetes in order to assess the predictability value of this metric in this context.Results: PE, along with some of its derivatives, was able to find significant differences between diabetic and non–diabetic patients from records acquired up to 3 years before the diagnosis. The quantitative results for PE were 3.5878 ± 0.3916 for the nondiabetic class, and 3.1564 ± 0.4166 for the diabetic class. With a classification accuracy higher than 70%, and by means of a Cox regression model, PE demonstrated that it is a very promising candidate as a risk stratification tool for continuous glucose monitoring.Conclusion: PE can be considered as a prospective tool for the early diagnosis of the glucoregulatory system.