Approximate Entropy Research Articles

EEG entropy insights in the context of physiological aging and Alzheimer's and Parkinson's diseases: a comprehensive review.

In recent decades, entropy measures have gained prominence in neuroscience due to the nonlinear behaviour exhibited by neural systems. This rationale justifies the application of methods from the theory of nonlinear dynamics to cerebral activity, aiming to detect and quantify its variability more effectively. In the context of electroencephalogram (EEG) signals, entropy analysis offers valuable insights into the complexity and irregularity of electromagnetic brain activity. By moving beyond linear analyses, entropy measures provide a deeper understanding of neural dynamics, particularly pertinent in elucidating the mechanisms underlying brain aging and various acute/chronic-progressive neurological disorders. Indeed, various pathologies can disrupt nonlinear structuring in neural activity, which may remain undetected by linear methods such as power spectral analysis. Consequently, the utilization of nonlinear tools, including entropy analysis, becomes crucial for capturing these alterations. To establish the relevance of entropy analysis and its potential to discern between physiological and pathological conditions, this review discusses its diverse applications in studying healthy brain aging and neurodegenerative diseases, including Alzheimer's disease (AD) and Parkinson's disease (PD). Various entropy parameters, such as approximate entropy (ApEn), sample entropy (SampEn), multiscale entropy (MSE), and permutation entropy (PermEn), are analysed within this context. By quantifying the complexity and irregularity of EEG signals, entropy analysis may serve as a valuable biomarker for early diagnosis, treatment monitoring, and disease management. Such insights offer clinicians crucial information for devising personalized treatment and rehabilitation plans tailored to individual patients.

Entropy of Neuronal Spike Patterns

Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content of neuronal patterns, entropy measures provide insights into neural coding strategies, synaptic plasticity, network dynamics, and cognitive processes. Here, we review basic entropy metrics and then we provide examples of recent advancements in using entropy as a tool to improve our understanding of neuronal processing. It focuses especially on studies on critical dynamics in neural networks and the relation of entropy to predictive coding and cortical communication. We highlight the necessity of expanding entropy measures from single neurons to encompass multi-neuronal activity patterns, as cortical circuits communicate through coordinated spatiotemporal activity patterns, called neuronal packets. We discuss how the sequential and partially stereotypical nature of neuronal packets influences the entropy of cortical communication. Stereotypy reduces entropy by enhancing reliability and predictability in neural signaling, while variability within packets increases entropy, allowing for greater information capacity. This balance between stereotypy and variability supports both robustness and flexibility in cortical information processing. We also review challenges in applying entropy to analyze such spatiotemporal neuronal spike patterns, notably, the “curse of dimensionality” in estimating entropy for high-dimensional neuronal data. Finally, we discuss strategies to overcome these challenges, including dimensionality reduction techniques, advanced entropy estimators, sparse coding schemes, and the integration of machine learning approaches. Thus, this work summarizes the most recent developments on how entropy measures contribute to our understanding of principles underlying neural coding.

A new spectral measure of complexity and its capabilities for detecting signals in noise

This article is devoted to the improvement of signal recognition methods based on the information characteristics of the spectrum. A discrete function of the normalized ordered spectrum is established for a single window function included in the DFT. Lemmas on estimates of entropy, imbalance and statistical complexity in processing a time series of independent Gaussian quantities are proved. New concepts of one-dimensional and two-dimensional spectral complexities are proposed. The theoretical results obtained were verified by numerical experiments, which confirmed the effectiveness of the new information characteristic when detecting a signal mixed with white noise at low signal-to-noise ratios.

Implementation of entropically secure encryption: Securing personal health data

AbstractEntropically secure encryption (ESE) offers unconditional security with shorter keys compared to the One‐Time Pad. Here, the first implementation of ESE for bulk encryption is presented. The main computational bottleneck for bulk ESE is a multiplication in a very large finite field. This involves multiplication of polynomials followed by modular reduction. A polynomial multiplication is implemented based on the gf2x library, with modifications that avoid inputs of vastly different length, thus improving speed. Additionally, a recently proposed efficient reduction algorithm that works for any polynomial degree is implemented. Two use cases are investigated: x‐ray images of patients and human genome data. Entropy estimation is conducted using compression methods whose results determine the key lengths required for ESE. The running times for all steps of the encryption are reported. The potential of ESE to be used in conjunction with quantum key distribution (QKD), in order to achieve full information‐theoretic security of QKD‐protected links for these use cases is discussed.

Shannon Wavelet‐Based Approximation Scheme for Information Entropy Integrals in Confined Domain

ABSTRACTIn this work, the author attempted to develop a Shannon wavelet‐based numerical scheme to approximate the information entropies in both configuration and momentum space corresponding to the ground and adjacent excited energy states of one‐dimensional Schrödinger equation appearing in non‐relativistic quantum mechanics. The development of this scheme is based on the judicious use of sinc scale functions as an approximation basis and a suitable numerical quadrature to approximate entropies in position and momentum spaces. Priori and posteriori errors appearing in the approximations of wave functions and entropy integrals have been discussed. The scheme (coded in Python) has been subsequently exercised for various exactly solvable and quasi‐exactly solvable non‐relativistic quantum mechanical models in confined domain.

Entropy-Based Volatility Analysis of Financial Log-Returns Using Gaussian Mixture Models

Volatility in financial markets refers to the variation in asset prices over time. High volatility indicates increased risk, making its evaluation essential for effective risk management. Various methods are used to assess volatility, with the standard deviation of log-returns being a common approach. However, this implicitly assumes that log-returns follow a Gaussian distribution, which is not always valid. In this paper, we explore the use of (differential) entropy to evaluate the volatility of financial log-returns. Estimation of entropy is obtained using a Gaussian mixture model to approximate the underlying density of log-returns. Following this modeling approach, popular risk measures such as Value at Risk and Expected Shortfall can also be computed. By integrating Gaussian mixture modeling and entropy into the analysis of log-returns, we aim to provide a more accurate and robust framework for assessing financial volatility and risk measures.

Heart rate variability and insomnia in depressed patients with breast cancer

ObjectivesDepression is associated with unhealthy autonomic regulation. However, whether patients with breast cancer (BC) with different degrees of depression can be identified from linear and non-linear dynamics in the autonomic...

Relaxed Equilibria for Time-Inconsistent Markov Decision Processes

This paper considers an infinite-horizon Markov decision process (MDP) that allows for general nonexponential discount functions in both discrete and continuous time. Because of the inherent time inconsistency, we look for a randomized equilibrium policy (i.e., relaxed equilibrium) in an intrapersonal game between an agent’s current and future selves. When we modify the MDP by entropy regularization, a relaxed equilibrium is shown to exist by a nontrivial entropy estimate. As the degree of regularization diminishes, the entropy-regularized MDPs approximate the original MDP, which gives the general existence of a relaxed equilibrium in the limit by weak convergence arguments. As opposed to prior studies that consider only deterministic policies, our existence of an equilibrium does not require any convexity (or concavity) of the controlled transition probabilities and reward function. Interestingly, this benefit of considering randomized policies is unique to the time-inconsistent case.

Quantifying Hand Motion Complexity in Simulated Sailing Using Inertial Sensors.

The control of hand movement during sailing is important for performance. To quantify the amount of regularity and the unpredictability of hand fluctuations during the task, the mathematical algorithm Approximate Entropy (ApEn) of the hand acceleration can be used. Approximate Entropy is a mathematical algorithm that depends on the combination of two input parameters including (1) the length of the sequences to be compared (m), and (2) the tolerance threshold for accepting similar patterns between two segments (r). The aim of this study is to identify the proper combinations of 'm' and 'r' parameter values for ApEn measurement in the hand movement acceleration data during sailing. Inertial Measurement Units (IMUs) recorded acceleration data for both the mainsail (non-dominant) and tiller (dominant) hands across the X-, Y-, and Z-axes, as well as vector magnitude. ApEn values were computed for 24 parameter combinations, with 'm' ranging from 2 to 5 and 'r' from 0.10 to 0.50. The analysis revealed significant differences in acceleration ApEn regularity between the two hands, particularly along the Z-axis, where the mainsail hand exhibited higher entropy values (p = 0.000673), indicating greater acceleration complexity and unpredictability. In contrast, the tiller hand displayed more stable and predictable acceleration patterns, with lower ApEn values. ANOVA results confirmed that parameter 'm' had a significant effect on acceleration complexity for both hands, highlighting differing motor control demands between the mainsail and tiller hands. These findings demonstrate the utility of IMU sensors and ApEn in detecting nuanced variations in acceleration dynamics during sailing tasks. This research contributes to the understanding of hand-specific acceleration patterns in sailing and provides a foundation for further studies on adaptive sailing techniques and motor control strategies for both novice and expert sailors.

Enhancing gait recognition in lower limb exoskeletons: adaptive feature selection and random forest with Bayesian optimization

Abstract To improve human-machine cooperation and enhance the accuracy of gait recognition in wearable lower limb exoskeletons, an enhancement method of gait recognition based on adaptive feature selection and an optimized machine learning algorithm was proposed. In this study, surface electromyography (sEMG) signals of rectus femoris, medialis femoris, lateralis femoris, semitendinosus and biceps femoris were recorded during level-ground walking. Then, time domain, frequency domain, time-frequency domain and nonlinear features were extracted. The integrated value of electromyography, variance, root mean square and wavelength were selected as the time domain features, and the mean power frequency and median frequency were selected as the frequency domain features. Wavelet packet energy was selected as the time-frequency domain feature. Nonlinear features including approximate entropy, sample entropy and fuzzy entropy of sEMG were extracted. To reduce feature dimension and improve the calculation efficiency, adaptive feature selection was performed by particle swarm optimization combined with sigmoid function. Then, the feature matrix was determined as the input for a random forest classifier to recognize different gait phases. An adaptive optimization mechanism based on Bayesian optimization was applied to random forest. Compared with random forest, the overall performance of the optimized model was improved. Its accuracy was increased by 3.57%. The feature selection and adaptive optimization mechanisms in gait recognition not only enhance the accuracy of random forest algorithms applied to sEMG for gait prediction, but also facilitate the flexibility and consistency required for lower limb exoskeleton gait control.

CONTINUOUS SCORING OF AUTISM SPECTRUM DISORDER PATIENTS BY ANALYZING THEIR EEG SIGNALS

Clinical manifestations and standard psychological tests have been widely used to diagnose autism spectrum disorder (ASD) patients and evaluate their severity level. The gold-standard criterion to diagnose ASD patients is the childhood autism rating scale (CARS), which is a qualitative questionnaire that is filled out through a systematic interview while no physiological test/record is performed to determine this score. To make the diagnosis process quantitative, electroencephalography (EEG) signals have been repeatedly analyzed to differentiate healthy subjects from ASD patients. However, the precise relationship between the abnormal behavior of EEG signals and different ASD severity levels is not well investigated. Here, we use CARS to qualitatively determine the severity level of 14 autistic children, who voluntarily enrolled in our study. We recorded their EEG signals from 19 scalp channels when they were awake in the idle state and elicited three informative features including approximation entropy, multiscale entropy and sample entropy in successive time frames. Among the three measures of entropy, the last one exhibits the highest sensitivity, where its correlation coefficient (CC) exceeds 0.7, on the electrode positions T6 (CC [Formula: see text] 0.74), P4 (CC [Formula: see text] 0.76) and Cz (CC [Formula: see text] 0.75). Results of sample entropy in channels Cz-versus-Pz and P4-versus-FP2 show that a simple K-nearest neighbor classifier can provide 93% classification accuracy among patients with mild, moderate and severe ASD levels. Comparing the proposed method to the conventional ones, we also extracted power spectral density features from the channels but they failed to identify the ASD severity level with an acceptable accuracy.

Key Rank Estimation Methods: Comparisons and Practical Considerations

New proposals for scalable key rank estimation methods have appeared recently, in particular the sampling based approach MCRank. The idea is that one can consistently estimate the key rank by sampling only a small portion of the key space as a “proxy”, leading to both an accurate and scalable approach, at least in comparison with another approach based on histograms. We show that the (earlier) GEEA algorithm is in fact a sampling based algorithm, and provide an in-depth comparison between GEEA (when adapted to produce rank estimates rather than guessing entropy estimates), GM bounds, MCRank and the currently most performant counting based rank estimation as implemented in the Labynkyr library. We find that although MCRank does live up to the promised accuracy and scalability for probability-based distinguishers, it fails to handle cases with unusual distinguisher distributions. Furthermore, we put forward a novel proposal for a highly scalable key rank estimation method by introducing the notion of an “attacker budget”. Our proposal is based on the idea that, in particular for very long keys, the exact key rank is less important than the knowledge whether a key is within a certain bound. Thus our “budget approach” is based on efficiently checking if the result of an attack is such that the attacker's budget suffices for successful enumeration. Our budget approach scales linearly with the key size and thus enables security estimations even for post-quantum key lengths.

Methods for faster estimation of the entropy profile of a lithium-ion battery: A comparison of accelerated potentiometry and the estimation of entropy through thermal signatures

The operating temperature of a lithium-ion battery (LIB) has strong effects on its electrochemical performance, safety, and lifetime. Reliable predictions of cell temperature and heat generation rate are required for the design and thermal management of battery systems. The entropy variation of a LIB has a significant influence on the heat generation and the cell temperature variation, especially at a lower C-rate. Since the entropy is a function of State-of-Charge (SoC), and may change during ageing, the entropy profile may as well be used as an indicator for battery State-of-Health diagnostics. The present experimental methods, such as potentiometric or calorimetric methods, for determining the entropy variation throughout a battery's full SoC window are time-consuming and require specialized equipment with good thermal insulation and instrumentation. Hence, faster and more convenient methods are needed. In this paper, we compare an accelerated potentiometric method with a new entropy characterization method where the entropy profile is extracted from the thermal signature of a LIB during charge and discharge at low C-rates. The accelerated potentiometric method aims to obtain the entropy from thermal cycling while the cell is still not completely relaxed to its stable OCV. The thermal signature method monitors the variations of a cell's outer surface temperature during a low constant current charge or discharge, from which the continuous variation of entropy with the SoC can be extracted. The obtained entropy profile was validated through a modeling approach that combines a P2D-electrochemical model for battery operation and a 3D-finite-element model for the thermal behavior during battery operation. Good agreement between the model predictions and experiments was achieved on the temporal variation of the cell's outer surface temperature.

Data-driven system identification and model predictive control of pneumatic conveying using nonlinear dynamics analysis for optimised energy consumption

Pneumatic conveying systems provide secure transportation of particulate material and a dust-free environment. These systems face high energy consumption, material degradation, and pipeline blockages. This research presents an innovative solution by integrating nonlinear dynamics analysis of electrostatic sensor data, including chaos and recurrence quantification analysis, sparse identification of nonlinear dynamics with control (SINDYc) and model predictive control (MPC). The Lyapunov exponent, approximate entropy and recurrence rate of electrostatic sensor data reveal the chaotic nature of gas-solid flows. MPC framework was tailored for real-time optimisation of a pneumatic conveying system. SINDYc system models were developed using data collected from an open-loop control pneumatic conveying process to conduct MPC simulations and select an appropriate model for real-time control. This research illustrates the potential of integrating nonlinear dynamics analysis, SINDYc integrated and MPC for enhanced system performance, showcased a significant reduction in energy consumption without compromising the system's efficiency or reliability.

Eversion Force Control and Neural Drive to the Peroneus Muscles in Male Collegiate Soccer Athletes With and Without Perceived Ankle Instability

OBJECTIVE: To investigate whether individuals with perceived ankle instability (PAI) exhibited lower force control capacity and higher neural drive variability during ankle eversion force control tests compared to individuals without PAI. DESIGN: Cross-sectional case-control study. METHODS: We included male collegiate soccer athletes with ( n = 16) and without PAI ( n = 11). Force steadiness and accuracy tests were performed at 10%, 30%, and 50% of maximal voluntary contraction. Neural drive to the peroneal muscles was calculated using high-density surface electromyography. We analyzed force error, force complexity, and force and neural drive variability. RESULTS: Eversion strength was higher in the PAI group (0.36 ± 0.1 N m/kg) ( P = .03). Force error (mean difference [MD], 0.05; 95% confidence interval [CI]: −0.31, 0.21), force complexity (approximate entropy) (MD, 0.08; 95% CI: − 0.34, 0.50), force (MD, 0.20; 95% CI: −1.77, 1.36), and neural drive (MD, 1.75; 95% CI: −2.48, 5.91) variability at 10% maximal voluntary contraction did not differ between groups ( P>.05). There were no significant differences between groups in other intensities and variables ( P>.05). CONCLUSION: Eversion force control and neural drive to the peroneal muscles in male collegiate soccer athletes with PAI did not differ from those in athletes without PAI. JOSPT Open 2024;2(4):315-321. Epub 15 July 2024. doi:10.2519/josptopen.2024.0360

High efficiency deep image compression via channel-wise scale adaptive latent representation learning

Recent learning based neural image compression methods have achieved impressive rate–distortion (RD) performance via the sophisticated context entropy model, which performs well in capturing the spatial correlations of latent features. However, due to the dependency on the adjacent or distant decoded features, existing methods require an inefficient serial processing structure, which significantly limits its practicability. Instead of pursuing computationally expensive entropy estimation, we propose to reduce the spatial redundancy via the channel-wise scale adaptive latent representation learning, whose entropy coding is spatially context-free and parallelizable. Specifically, the proposed encoder adaptively determines the scale of the latent features via a learnable binary mask, which is optimized with the RD cost. In this way, lower-scale latent representation will be allocated to the channels with higher spatial redundancy, which consumes fewer bits and vice versa. The downscaled latent features could be well recovered with a lightweight inter-channel upconversion module in the decoder. To compensate for the entropy estimation performance degradation, we further develop an inter-scale hyperprior entropy model, which supports the high efficiency parallel encoding/decoding within each scale of the latent features. Extensive experiments are conducted to illustrate the efficacy of the proposed method. Our method achieves bitrate savings of 18.23%, 19.36%, and 27.04% over HEVC Intra, along with decoding speeds that are 46 times, 48 times, and 51 times faster than the baseline method on the Kodak, Tecnick, and CLIC datasets, respectively.

Enhance Key Stage Generation for Developing Kasumi Encryption Algorithm

Today, data security is a critical component of the efficient performance of every organization's diverse requirement, one of the important requirements is to provide a secure connection for data transmission in organization's networks. Cryptography algorithms are used to protect transmitted data from unauthorized access. Every day, a great deal of research is being done. The process of encrypting data is currently under development. Therefore, a substantial technological or research effort is still required. Cellular networks and secure communication. This proposal calls for the development of a new encryption system based on Using random keys in all the rounds. That’s mean we generated random keys as 128 bits for each round and save it in an array with size (8×8) because we have 8 rounds and each key divided into 8 sub keys as 16 bits for each to use in encryption and decryption. The developed random keys and traditional algorithm keys were tested with statistical tests of NIST, which made up a comparison and conclusion that the developed keys have a higher randomness specially in linear complexity, Frequency within block and random excursion tests, which leads to a better quality of encryption. Then we could test the cipher text with frequency within block test that was evaluated by (0.49590998713105255) and the approximate entropy test that was evaluated by (0.4922198616875501) for each cipher text which resulted from standard and modified Kasumi algorithm that’s means we can conclude that our modified Kasumi algorithm achieved more secure data in its resulted cipher text.

A three-dimensional discrete fractional-order HIV-1 model related to cancer cells, dynamical analysis and chaos control

In this paper, we study a three-dimensional discrete-time model to describe the behavior of cancer cells in the presence of healthy cells and HIV-infected cells. Based on the Caputo-like difference operator, we construct the fractional-order biological system. This study's significance lies in developing a new approach to presenting a biological dynamical system. Since the qualitative analysis related to existence, uniqueness, and stability is almost the same as can be found in numerous existing papers, and comparing this study to other research, constructing a biological discrete system using the Caputo difference operator can be particularly important. Using powerful tools of nonlinear theory such as phase plots, bifurcation diagrams, Lyapunov exponent spectrum, and the 0-1 test, we establish that the proposed system can exhibit different biological states, including stable, periodic, and chaotic behaviors. Here, the route leading to chaos is period-doubling bifurcation. Furthermore, the level of chaos in the system is quantified using $C_{0}$ complexity and approximate entropy algorithms. The stabilization or suppression of chaotic motions in the fractional-order system is presented, where an efficient controller is designed based on the stability theory of the discrete-time fractional-order systems. Numerical simulations are provided to validate the theoretical results derived in this research paper.

Enhance Key Stage Generation for Developing Kasumi Encryption Algorithm

Today, data security is a critical component of the efficient performance of every organization's diverse requirement, one of the important requirements is to provide a secure connection for data transmission in organization's networks. Cryptography algorithms are used to protect transmitted data from unauthorized access. Every day, a great deal of research is being done. The process of encrypting data is currently under development. Therefore, a substantial technological or research effort is still required. Cellular networks and secure communication. This proposal calls for the development of a new encryption system based on Using random keys in all the rounds. That’s mean we generated random keys as 128 bits for each round and save it in an array with size (8×8) because we have 8 rounds and each key divided into 8 sub keys as 16 bits for each to use in encryption and decryption. The developed random keys and traditional algorithm keys were tested with statistical tests of NIST, which made up a comparison and conclusion that the developed keys have a higher randomness specially in linear complexity, Frequency within block and random excursion tests, which leads to a better quality of encryption. Then we could test the cipher text with frequency within block test that was evaluated by (0.49590998713105255) and the approximate entropy test that was evaluated by (0.4922198616875501) for each cipher text which resulted from standard and modified Kasumi algorithm that’s means we can conclude that our modified Kasumi algorithm achieved more secure data in its resulted cipher text.

A Lightweight Multi-Mental Disorders Detection Method Using Entropy-Based Matrix from Single-Channel EEG Signals.

Background: Mental health issues are increasingly prominent worldwide, posing significant threats to patients and deeply affecting their families and social relationships. Traditional diagnostic methods are subjective and delayed, indicating the need for an objective and effective early diagnosis method. Methods: To this end, this paper proposes a lightweight detection method for multi-mental disorders with fewer data sources, aiming to improve diagnostic procedures and enable early patient detection. First, the proposed method takes Electroencephalography (EEG) signals as sources, acquires brain rhythms through Discrete Wavelet Decomposition (DWT), and extracts their approximate entropy, fuzzy entropy, permutation entropy, and sample entropy to establish the entropy-based matrix. Then, six kinds of conventional machine learning classifiers, including Support Vector Machine (SVM), k-Nearest Neighbors (kNN), Naive Bayes (NB), Generalized Additive Model (GAM), Linear Discriminant Analysis (LDA), and Decision Tree (DT), are adopted for the entropy-based matrix to achieve the detection task. Their performances are assessed by accuracy, sensitivity, specificity, and F1-score. Concerning these experiments, three public datasets of schizophrenia, epilepsy, and depression are utilized for method validation. Results: The analysis of the results from these datasets identifies the representative single-channel signals (schizophrenia: O1, epilepsy: F3, depression: O2), satisfying classification accuracies (88.10%, 75.47%, and 89.92%, respectively) with minimal input. Conclusions: Such performances are impressive when considering fewer data sources as a concern, which also improves the interpretability of the entropy features in EEG, providing a reliable detection approach for multi-mental disorders and advancing insights into their underlying mechanisms and pathological states.