Approximate Entropy Research Articles

To Study the Use of Entropy and the Preprocessing Step in Analyzing Web Users' Behavior and Improving Web Mining

The study of entropy's application to online use mining is the main goal of the work. By using the Reference Length approach, entropy provides an additional option for figuring out the ratio of auxiliary pages in the session identification. Two distinct online portals were used for the experiment. The first log file was downloaded via the web portal of a virtual learning environment course. The online gateway with anonymous access provided the second log file. When entropy and sitemap estimations of the ratio of auxiliary pages were compared, it was discovered that entropy could replace the sitemap estimate of the ratio of auxiliary pages with a full-valued substitute in the situation of sitemap abundance.

Effects of training repetition on young footballers’ tactical and physical performances: free and conditioned large-sided games

This study evaluated the consequences of repeating the same large-sided games, playing free or conditioned, during several training sessions on young footballers’ tactical and conditional performances and their variabilities. Thirty-two U14 and U16 developmental male footballers participated in the study. Both teams, divided into two balanced groups (free play and conditioned), faced each other during three eight-a-side games (seven vs seven, plus goalkeepers) during three training sessions. The free-play groups played as they pleased, while the conditioned groups played the games conditioned by tactical instructions. Tactical performance was assessed by central tendency and normalized approximate entropy measures of the distance of each player to the team centroid (m) and the distance of each player to their mean position (m), while the conditional performance was assessed by total distance (m) and walking, jogging, running, and sprinting distances traveled (m), and number of accelerations and decelerations performed. Whereas players’ average tactical and conditional responses scarcely varied between sessions, inter- and intra-player variabilities were always considerable (coefficient of variation >10%) regardless of the task conditions and the age group. In addition, all footballers’ conditional variability was substantially higher at faster speeds (jogging, running, and sprinting distances) than at slower ones (total distance and walking distance traveled), and for accelerations and decelerations performed. Implementing large-sided games with or without tactical instructions may be an appropriate training strategy to ensure stable and constant average responses for the team. Nevertheless, academy football coaches should bear in mind and manage the variability between and within players.

Chaotic systems based on higher-order oscillatory equations

This paper discusses the design process toward new lumped chaotic systems that originates in higher-order ordinary differential equations commonly used as description of ideal oscillators. In investigated third-order case, two chaotic oscillators were constructed. These systems are dual in the sense of vector field geometry local to fixed points. The existence of robust chaos was proved by both standard routines of numerical analysis and practical measurement. For the fourth-order oscillatory equation, the concept based on interaction between superinductor and supercapacitor was examined in detail. Since both “superelements” are active, the nonlinearity essential to the evolution of chaos is fully passive. It is demonstrated that complex motion is robust and does not represent long transient behavior or numerical artefact. The existence of chaos was verified using standard quantifiers of the flow, such as the largest Lyapunov exponents, recurrence plots, approximate entropy and sensitivity calculation. A good final agreement between theoretical assumptions and practical results will be concluded, on a visual comparison basis.

Comprehensive linear and nonlinear heart rate variability normative data in children.

The autonomic nervous system (ANS) is critical in regulating involuntary bodily functions, including heart rate. Heart rate variability (HRV) reflects the complex interplay between the ANS and humoral factors, making it a valuable noninvasive tool for assessing autonomic function. While HRV has been extensively studied in adults, normative data for HRV in children, primarily based on long-term rhythm recordings, are limited. This study aimed to establish comprehensive normative data for HRV in children. In this retrospective study, we examined 24-h Holter monitors of children aged 1day to 18years, divided into six age groups, at Nemours Children's Health in Orlando, Florida, spanning the years 2013-2023. HRV analysis encompassed time-domain, frequency-domain, and nonlinear indices. Holter data for a total of 247 patients in six age groups were included. An age-related uptrend was observed in all time- and frequency-domain variables except the normalized unit of low-frequency power. Entropy analysis revealed contradictory results among different entropy techniques. Sample and approximate entropy analyses were consistent and showed less complexity and more predictability of HRV with decreasing heart rate, while Shannon entropy analysis showed the opposite. Fractal detrended fluctuation analysis exhibited significant decreases across the age groups, suggestive of diminishing self-similarity of HRV patterns. Control of heart rate and HRV is a highly complex process and requires further study for a better understanding. It seems that no single parameter can fully elucidate the entire process. A combination of time-domain, frequency-domain, and nonlinear indices may be necessary to explain HRV behavior in the growing body.

Extropy and entropy estimation based on progressive Type-I interval censoring

This paper proposes nonparametric estimates for the two information measures extropy and entropy when a progressively Type-I interval censored data is available. Different nonparametric approaches are used for deriving the estimates, including: moments of the empirical cumulative distribution function and linear regression. The performance of the proposed estimates is studied under various censoring schemes via simulation studies. Furthermore, different real data sets are analyzed for illustrative purposes.The estimates based on linear approximation J^2 and H^2 outperform the other estimate in the majority of studied cases.

The impact of controlled breathing on autonomic nervous system modulation: analysis using phase-rectified signal averaging, entropy and heart rate variability

Objective.The present study investigated how breathing stimuli affect both non-linear and linear metrics of the autonomic nervous system (ANS).Approach.The analysed dataset consisted of 70 young, healthy volunteers, in whom arterial blood pressure (ABP) was measured noninvasively during 5 min sessions of controlled breathing at three different frequencies: 6, 10 and 15 breaths min-1. CO2concentration and respiratory rate were continuously monitored throughout the controlled breathing sessions. The ANS was characterized using non-linear methods, including phase-rectified signal averaging (PRSA) for estimating heart acceleration and deceleration capacity (AC, DC), multiscale entropy, approximate entropy, sample entropy, and fuzzy entropy, as well as time and frequency-domain measures (low frequency, LF; high-frequency, HF; total power, TP) of heart rate variability (HRV).Main results.Higher breathing rates resulted in a significant decrease in end-tidal CO2concentration (p< 0.001), accompanied by increases in both ABP (p <0.001) and heart rate (HR,p <0.001). A strong, linear decline in AC and DC (p <0.001 for both) was observed with increasing breathing rate. All entropy metrics increased with breathing frequency (p <0.001). In the time-domain, HRV metrics significantly decreased with breathing frequency (p <0.01 for all). In the frequency-domain, HRV LF and HRV HF decreased (p= 0.038 andp= 0.040, respectively), although these changes were modest. There was no significant change in HRV TP with breathing frequencies.Significance.Alterations in CO2levels, a potent chemoreceptor trigger, and changes in HR most likely modulate ANS metrics. Non-linear PRSA and entropy appear to be more sensitive to breathing stimuli compared to frequency-dependent HRV metrics. Further research involving a larger cohort of healthy subjects is needed to validate our observations.

Maze based image encryption method constructed by random number generation

Due to the large size of image files and the high correlation between the numerical values of the pixels, various improved algorithms have been used for image encryption in recent years. In most cases improvement is obtained by taking into account the characteristics of chaotic and random processes in encryption algorithms. In this study, an algorithm is proposed that encrypts images based on a maze created by keys consisting of random numbers generated with Feigembaum’s quadratic function. For the generation of the maze, deep first search (DFS) algorithm has been used. During the generation with this algorithm, numbers representing the walking paths and as well as the length of the used stack are included in the encryption process. Algorithm security verification was carried out according to key space and differential analysis, statistical analysis such as entropy estimation, and analysis of correlation between the histogram and neighboring pixels of the image.

Estimation of average differential entropy for a stationary ergodic space-time random field on a bounded area

Estimation of average differential entropy for a stationary ergodic space-time random field on a bounded area

Drought prediction in Jilin Province based on deep learning and spatio-temporal sequence modeling

Jilin Province, a key agricultural hub in Northeast China, has long been impacted by climate change, with drought disasters significantly affecting its agricultural output and ecological environment. Accurate drought prediction is essential for the effective utilization of water resources and agricultural production. This study proposes a novel drought prediction model that utilizes the SSA-VMD (Sparrow Search Algorithm Optimized Variational Mode Decomposition) technique to decompose meteorological data, followed by the reconstruction of the decomposed components using four entropy algorithms, including approximate entropy. The model integrates ARIMA (AutoRegressive Integrated Moving Average) and BiLSTM (Bidirectional Long Short-Term Memory network) for forecasting the reconstructed components, subsequently combining their outputs. In terms of spatio-temporal data, the study employs prediction models including the spatio-temporal cube, spatio-temporal hotspot analysis integrated with empirical kriging, and local outlier analysis to examine spatial distribution. The model’s predictive performance is validated from three perspectives: statistical characteristics of the indicators, comparison between predicted and observed values through prediction curve plots, and box plots. The results demonstrate that the combined SSA-VMD-ARIMA-BiLSTM model significantly enhances prediction accuracy compared to single models, as exemplified by its application in Changchun City. The model achieved an R2 of 0.938 and a root mean square error (RMSE) of 0.047 in drought prediction, outperforming the single ARIMA model (R2: 0.636, RMSE: 0.709) and the BiLSTM model (R2: 0.514, RMSE: 0.901). Additionally, across the entire province, the model’s R2, MAE, and RMSE are 0.82, 0.15, and 0.083, respectively, suggesting that the model exhibits not only high prediction accuracy but also a degree of generalizability. Furthermore, the results from the spatio-temporal cube, spatio-temporal hotspot analysis, and local outlier analysis demonstrate the method’s high accuracy and stability in predicting both short-term and long-term droughts. Particularly in short-term drought prediction, the model effectively captures the spatio-temporal distribution characteristics of short-term meteorological droughts. This study offers new methodological support for enhancing the early warning capabilities of drought risk in Jilin Province, providing a robust foundation for addressing the challenges posed by climate change. The findings not only address certain shortcomings in current drought prediction research but also introduce new methodologies and perspectives for future studies.

Many-Body Entropies and Entanglement from Polynomially Many Local Measurements

Estimating global properties of many-body quantum systems such as entropy or bipartite entanglement is a notoriously difficult task, typically requiring a number of measurements or classical postprocessing resources growing exponentially in the system size. In this work, we address the problem of estimating global entropies and mixed-state entanglement via partial-transposed (PT) moments and show that efficient estimation strategies exist under the assumption that all the spatial correlation lengths are finite. Focusing on one-dimensional systems, we identify a set of approximate factorization conditions (AFCs) on the system density matrix, which allow us to reconstruct entropies and PT moments from information on local subsystems. This identification yields a simple and efficient strategy for entropy and entanglement estimation. Our method could be implemented in different ways, depending on how information on local subsystems is extracted. Focusing on randomized measurements providing a practical and common measurement scheme, we prove that our protocol requires only polynomially many measurements and postprocessing operations, assuming that the state to be measured satisfies the AFCs. We prove that the AFCs hold for finite-depth quantum-circuit states and translation-invariant matrix-product density operators and provide numerical evidence that they are satisfied in more general, physically interesting cases, including thermal states of local Hamiltonians. We argue that our method could be practically useful to detect bipartite mixed-state entanglement for large numbers of qubits available in today’s quantum platforms. Published by the American Physical Society 2024

Sharma–Taneja–Mittal Entropy and Its Application of Obesity in Saudi Arabia

This paper presents several nonparametric estimators for the Sharma–Taneja–Mittal entropy measure of a continuous random variable with known support, utilizing spacing, a local linear model, and a kernel function. The properties of these estimators are discussed. Their performance was also examined through real data analysis and Monte Carlo simulations. In the Monte Carlo experiments, the proposed Sharma–Taneja–Mittal entropy estimators were employed to create a test of goodness-of-fit under the standard uniform distribution. The suggested test statistics demonstrate strong performance, as evidenced by a comparison of their power with that of other tests for uniformity. Finally, we examine a classification issue in the recognition of patterns to underscore the significance of these measures.

Determining the Optimal Window Duration to Enhance Emotion Recognition Based on Galvanic Skin Response and Photoplethysmography Signals

Automatic emotion recognition using portable sensors is gaining attention due to its potential use in real-life scenarios. Existing studies have not explored Galvanic Skin Response and Photoplethysmography sensors exclusively for emotion recognition using nonlinear features with machine learning (ML) classifiers such as Random Forest, Support Vector Machine, Gradient Boosting Machine, K-Nearest Neighbor, and Decision Tree. In this study, we proposed a genuine window sensitivity analysis on a continuous annotation dataset to determine the window duration and percentage of overlap that optimize the classification performance using ML algorithms and nonlinear features, namely, Lyapunov Exponent, Approximate Entropy, and Poincaré indices. We found an optimum window duration of 3 s with 50% overlap and achieved accuracies of 0.75 and 0.74 for both arousal and valence, respectively. In addition, we proposed a Strong Labeling Scheme that kept only the extreme values of the labels, which raised the accuracy score to 0.94 for arousal. Under certain conditions mentioned, traditional ML models offer a good compromise between performance and low computational cost. Our results suggest that well-known ML algorithms can still contribute to the field of emotion recognition, provided that window duration, overlap percentage, and nonlinear features are carefully selected.

Schizophrenia diagnosis based on diverse epoch size resting-state EEG using machine learning.

Schizophrenia is a severe mental disorder that impairs a person's mental, social, and emotional faculties gradually. Detection in the early stages with an accurate diagnosis is crucial to remedying the patients. This study proposed a new method to classify schizophrenia disease in the rest state based on neurologic signals achieved from the brain by electroencephalography (EEG). The datasets used consisted of 28 subjects, 14 for each group, which are schizophrenia and healthy control. The data was collected from the scalps with 19 EEG channels using a 250 Hz frequency. Due to the brain signal variation, we have decomposed the EEG signals into five sub-bands using a band-pass filter, ensuring the best signal clarity and eliminating artifacts. This work was performed with several scenarios: First, traditional techniques were applied. Secondly, augmented data (additive white Gaussian noise and stretched signals) were utilized. Additionally, we assessed Minimum Redundancy Maximum Relevance (MRMR) as the features reduction method. All these data scenarios are applied with three different window sizes (epochs): 1, 2, and 5 s, utilizing six algorithms to extract features: Fast Fourier Transform (FFT), Approximate Entropy (ApEn), Log Energy entropy (LogEn), Shannon Entropy (ShnEn), and kurtosis. The L2-normalization method was applied to the derived features, positively affecting the results. In terms of classification, we applied four algorithms: K-nearest neighbor (KNN), support vector machine (SVM), quadratic discriminant analysis (QDA), and ensemble classifier (EC). From all the scenarios, our evaluation showed that SVM had remarkable results in all evaluation metrics with LogEn features utilizing a 1-s window size, impacting the diagnosis of Schizophrenia disease. This indicates that an accurate diagnosis of schizophrenia can be achieved through the right features and classification model selection. Finally, we contrasted our results to recently published works using the same and a different dataset, where our method showed a notable improvement.

Wind Turbine Blade Fault Diagnosis: Approximate Entropy as a Tool to Detect Erosion and Mass Imbalance

Wind energy is a clean, sustainable, and renewable source. It is receiving a large amount of attention from governments and energy companies worldwide as it plays a significant role as an alternative source of energy in reducing carbon emissions. However, due to long-term operation in reduced and difficult weather conditions, wind turbine blades are always seriously damaged. Hence, damage detection in blade structure is essential to evaluate its operational condition and ensure its structural integrity and safety. We aim to use fractal, entropy, and chaos concepts as descriptors for the diagnosis of wind turbine blade condition. They are, respectively, estimated by the correlation dimension, approximate entropy, and the Lyapunov exponent. Formal statistical tests are performed to check how they are different across wind turbine blade conditions. The experimental results follow. First, the correlation dimension is not able to distinguish between all conditions of wind turbine blades. Second, approximate entropy is suitable to distinguish between healthy and erosion conditions and between healthy and mass imbalance conditions. Third, chaos is not a discriminative feature to distinguish between wind turbine blade conditions. Fourth, wind turbine blades with either erosion or mass imbalance exhibit less irregularity in their respective signals than healthy wind turbine blades.

Chaotic nature of the electroencephalogram during shallow and deep anesthesia: From analysis of the Lyapunov exponent

In conscious states, the electrodynamics of the cortex are reported to work near a critical point or phase transition of chaotic dynamics, known as the edge-of-chaos, representing a boundary between stability and chaos. Transitions away from this boundary disrupt cortical information processing and induce a loss of consciousness. The entropy of the electroencephalogram (EEG) is known to decrease as the level of anesthesia deepens. However, whether the chaotic dynamics of electroencephalographic activity shift from this boundary to the side of stability or the side of chaotic enhancement during anesthesia-induced loss of consciousness remains poorly understood. We investigated the chaotic properties of EEGs at two different depths of clinical anesthesia using the maximum Lyapunov exponent, which is mathematically regarded as a formal measure of chaotic nature, using the Rosenstein algorithm. In 14 adult patients, 12 s of electroencephalographic signals were selected during two depths of clinical anesthesia (sevoflurane concentration 2% as relatively deep anesthesia, sevoflurane concentration 0.6% as relatively shallow anesthesia). Lyapunov exponents, correlation dimensions and approximate entropy were calculated from these electroencephalographic signals. As a result, maximum Lyapunov exponent was generally positive during sevoflurane anesthesia, and both maximum Lyapunov exponents and correlation dimensions were significantly greater during deep anesthesia than during shallow anesthesia despite reductions in approximate entropy. The chaotic nature of the EEG might be increased at clinically deeper inhalational anesthesia, despite the decrease in randomness as reflected in the decreased entropy, suggesting a shift to the side of chaotic enhancement under anesthesia.

Ultrasound normalized cumulative residual entropy imaging: Theory, methodology, and application

Background and Objective:Ultrasound information entropy imaging is an emerging quantitative ultrasound technique for characterizing local tissue scatterer concentrations and arrangements. However, the commonly used ultrasound Shannon entropy imaging based on histogram-derived discrete probability estimation suffers from the drawbacks of histogram settings dependence and unknown estimator performance. In this paper, we introduced the information-theoretic cumulative residual entropy (CRE) defined in a continuous distribution of cumulative distribution functions as a new entropy measure of ultrasound backscatter envelope uncertainty or complexity, and proposed ultrasound CRE imaging for tissue characterization. Methods:We theoretically analyzed the CRE for Rayleigh and Nakagami distributions and proposed a normalized CRE for characterizing scatterer distribution patterns. We proposed a method based on an empirical cumulative distribution function estimator and a trapezoidal numerical integration for estimating the normalized CRE from ultrasound backscatter envelope signals. We presented an ultrasound normalized CRE imaging scheme based on the normalized CRE estimator and the parallel computation technique. We also conducted theoretical analysis of the differential entropy which is an extension of the Shannon entropy to a continuous distribution, and introduced a method for ultrasound differential entropy estimation and imaging. Monte-Carlo simulation experiments were performed to evaluate the estimation accuracy of the normalized CRE and differential entropy estimators. Phantom simulation and clinical experiments were conducted to evaluate the performance of the proposed normalized CRE imaging in characterizing scatterer concentrations and hepatic steatosis (n = 204), respectively. Results:The theoretical normalized CRE for the Rayleigh distribution was π/4, corresponding to the case where there were ≥10 randomly distributed scatterers within the resolution cell of an ultrasound transducer. The theoretical normalized CRE for the Nakagami distribution decreased as the Nakagami parameter m increased, corresponding to that the ultrasound backscattered statistics varied from pre-Rayleigh to Rayleigh and to post-Rayleigh distributions. Monte-Carlo simulation experiments showed that the proposed normalized CRE and differential entropy estimators can produce a satisfying estimation accuracy even when the size of the test samples is small. Phantom simulation experiments showed that the proposed normalized CRE and differential entropy imaging can characterize scatterer concentrations. Clinical experiments showed that the proposed ultrasound normalized CRE imaging is capable to quantitatively characterize hepatic steatosis, outperforming ultrasound differential entropy imaging and being comparable to ultrasound Shannon entropy and Nakagami imaging. Conclusion:This study sheds light on the theory and methodology of ultrasound normalized CRE. The proposed ultrasound normalized CRE can serve as a new, flexible quantitative ultrasound envelope statistics parameter. The proposed ultrasound normalized CRE imaging may find applications in quantified characterization of biological tissues. Our code will be made available publicly at https://github.com/zhouzhuhuang.

High-throughput true random number generator based on a dual-input oscillation circuit

The rapid development in the fields of information encryption, key generation, and algorithm initialization requires an increasing rate of true random numbers, making it necessary to design a fast true random number generator. In this work, a dual-mode conversion high throughput true random number generator (TRNG) based on a dual input oscillating XOR circuit (DIO-XOR) is proposed, which is designed using DIO-XOR, combined with a MUX selects the feedback mode of the whole circuit: self-feedback/mutual feedback. It is verified by automatic layout and routing on Xilinx Artix-7 and Kintex-7 FPGAs with a throughput rate of 650Mbps, which is achieved with low hardware overhead, and passes the entropy estimation test suite, NIST SP800-90B, as well as TESTU01 with high entropy, the sequences generated in the temperature-voltage fluctuation test pass the NIST SP800-22 test with good robustness.

Accurate estimation of the normalized mutual information of multidimensional data.

While the linear Pearson correlation coefficient represents a well-established normalized measure to quantify the inter-relation of two stochastic variables X and Y, it fails for multidimensional variables, such as Cartesian coordinates. Avoiding any assumption about the underlying data, the mutual information I(X, Y) does account for multidimensional correlations. However, unlike the normalized Pearson correlation, it has no upper bound (I ∈ [0, ∞)), i.e., it is not clear if say, I = 0.4 corresponds to a low or a high correlation. Moreover, the mutual information (MI) involves the estimation of high-dimensional probability densities (e.g., six-dimensional for Cartesian coordinates), which requires a k nearest-neighbor algorithm, such as the estimator by Kraskov et al. [Phys. Rev. E 69, 066138 (2004)]. As existing methods to normalize the MI cannot be used in connection with this estimator, a new approach is presented, which uses an entropy estimation method that is invariant under variable transformations. The algorithm is numerically efficient and does not require more effort than the calculation of the (un-normalized) MI. After validating the method by applying it to various toy models, the normalized MI between the Cα-coordinates of T4 lysozyme is considered and compared to a correlation analysis of inter-residue contacts.

Unsteady radiative-convective Casson hybrid nanofluid flow over an inclined disk with Cattaneo–Christov heat flux and entropy estimation

Unsteady radiative-convective Casson hybrid nanofluid flow over an inclined disk with Cattaneo–Christov heat flux and entropy estimation

New computations of the fractional worms transmission model in wireless sensor network in view of new integral transform with statistical analysis; an analysis of information and communication technologies

Wireless sensor networks (WSNs) have attracted a lot of interest due to their enormous potential for both military and civilian uses. Worm attacks can quickly target WSNs because of the network's weak security. The worm can spread throughout the network by interacting with a single unsafe node. Moreover, the analysis of worm spread in WSNs can benefit from the use of mathematical epidemic models. We suggest a five-compartment model to characterize the mechanisms of worm proliferation with respect to time in WSN. Taking into account the ZZ transform convoluted with the Atangana-Baleanu-Caputo (ABC) fractional derivative operator, we employ it to analyze the characteristics and applications of the ZZ transformation using the Mittag-Leffler kernel. Moreover, we construct a new algorithm for the homotopy perturbation method (HPM) in conjunction with the ZZ transform technique to generate analytical solutions for the worm transmission model. Also, we address the qualitative aspects such as positivity, boundness, worm-free state, endemic state, basic reproduction number (R0) and worm-free equilibrium stability. Furthermore, we prove that the virus rate in sensor nodes is extinct if R0<1 and the virus persists if R0>1. In addition, we develop analytical findings to evaluate the series of solutions. Furthermore, a detailed statistical analysis is conducted to verify the nonlinear dynamics of the system by verifying the 0−1 test to determine whether uncertainty exists using approximation entropy and the C0 data. An extensive analysis of the vaccination class with respect to the transmitting class as well as the susceptible class is being used to investigate the effects of stepping up precautions on WP in WSN. Moreover, the modeling of the WSN revealed that reducing the fractional-order from 1 requires that the recommended approach be implemented at the highest rate so that there is no long-lasting immunization; instead, nodes remain briefly defensive before becoming vulnerable to future worm attacks.