Approximate Entropy Research Articles

Deep Learning Model for Prediction of Dementia Severity based on EEG Signals

This study aimed to determine variations in the electroencephalograms (EEGs) of 15 individuals who were diagnosed with mild cognitive impairment (MCI) following stroke, 5 individuals suffering from vascular dementia (VD) and 15 healthy normal control (NC) individuals who performed a working memory task. Conventional filters including notch and bandpass filters were utilised to remove noise from the EEG data. The proposed method comprises computing the estimates of the attention entropy (AttEn), bubble entropy (BubbEn) and symbolic dynamic entropy (SyDyEn) of univariate data sequence features. The long short-term memory (LSTM) deep learning neural network was used to automatically classify dementia severity through noninvasive EEG-based recordings. The LSTM classification result with AttEn exceeds an average of 88.9% than BubbEn and SyDyEn, with classification results of 69.2% and 77.7%, respectively. The analysis of the brain EEG-based dementia severity dataset suggests that AttEn could potentially serve as a biomarker for detecting dementia severity. AttEn can capture relevant patterns and features in the EEG data and may be indicative of the severity of dementia with LSTM RNN to differentiate patients with VD, patients with MCI and NC individuals.

[formula omitted]-wave-like properties for entropy solutions to scalar parabolic–hyperbolic conservation laws

In this paper, we consider qualitative properties for entropy solutions to one-dimensional Cauchy problems (CP) for scalar parabolic–hyperbolic conservation laws. Since the equations have both properties of hyperbolic equations and those of parabolic equations, it is difficult to investigate the behavior of solutions to (CP). In our previous works, we focused on the traveling wave structure instead of the self-similar structure. In fact, we succeeded in constructing shock wave type traveling waves with multiple discontinuity. Moreover, we constructed rarefaction wave type sub-, super-solutions to (CP) and investigated their properties.In the present paper, we investigate “N-wave-like properties” for entropy solutions to (CP) while we are not able to construct an analogue of N-waves. In particular, we derive generalized one-sided Lipschitz estimates (Oleinik type entropy estimates) and decay estimates for entropy solutions to (CP). Based on the decay estimates, we discuss the asymptotic profiles of entropy solutions to (CP) under some specific setting.

How Gait Nonlinearities in Individuals Without Known Pathology Describe Metabolic Cost During Walking Using Artificial Neural Network and Multiple Linear Regression

This study uses Artificial Neural Networks (ANNs) and multiple linear regression (MLR) models to explore the relationship between gait dynamics and the metabolic cost. Six nonlinear metrics—Lyapunov Exponents based on Rosenstein’s algorithm (LyER), Detrended Fluctuation Analysis (DFA), the Approximate Entropy (ApEn), the correlation dimension (CD), the Sample Entropy (SpEn), and Lyapunov Exponents based on Wolf’s algorithm (LyEW)—were utilized to predict the metabolic cost during walking. Time series data from 10 subjects walking under 13 conditions, with and without hip exoskeletons, were analyzed. Six ANN models, each corresponding to a nonlinear metric, were trained using the Levenberg–Marquardt backpropagation algorithm and compared with MLR models. Performance was assessed based on the mean squared error (MSE) and correlation coefficients. ANN models outperformed MLR, with DFA and Lyapunov Exponent models showing higher R2 values, indicating stronger predictive accuracy. The results suggest that gait’s nonlinear characteristics significantly impact the metabolic cost, and ANNs are more effective for analyzing these dynamics than MLR models. The study emphasizes the potential of focusing on specific nonlinear gait variables to enhance assistive device optimization, particularly for hip exoskeletons. These findings support the development of personalized interventions that improve walking efficiency and reduce metabolic demands, offering insights into the design of advanced assistive technologies.

Fixed frequency range empirical wavelet transform based acoustic and entropy features for speech emotion recognition

The primary goal of automated speech emotion recognition (SER) is to accurately and effectively identify each specific emotion conveyed in a speech signal utilizing machines such as computers and mobile devices. The widespread recognition of the popularity of SER among academics for three decades is mainly attributed to its broad application in practical scenarios. The utilization of SER has proven to be beneficial in various fields, such as medical intervention, bolstering safety strategies, conducting vigil functions, enhancing online search engines, enhancing road safety, managing customer relationships, strengthening the connection between machines and humans, and numerous other domains. Many researchers have used diverse methodologies, such as the integration of different attributes, the use of different feature selection techniques, and designed a hybrid or complex model using more than one classifier, to augment the effectiveness of emotion classification. In our study, we used a novel technique called the fixed frequency range empirical wavelet transform (FFREWT) filter bank decomposition method to extract the features, and then used those features to accurately identify each and every emotion in the speech signal. The FFREWT filter bank method segments the speech signal frame (SSF) into many sub-signals or modes. We used each FFREWT-based decomposed mode to get features like the mel frequency cepstral coefficient (MFCC), approximate entropy (ApEn), permutation entropy (PrEn), and increment entropy (IrEn). We then used the different combinations of the proposed FFREWT-based feature sets and the deep neural network (DNN) classifier to classify the speech emotion. Our proposed method helps to achieve an emotion classification accuracy of 89.35%, 84.69%, and 100% using the combinations of the proposed FFREWT-based feature (MFCC + ApEn + PrEn + IrEn) for the EMO-DB, EMOVO, and TESS datasets, respectively. Our experimental results were compared with the other methods, and we found that the proposed FFREWT-based feature combinations with a DNN classifier performed better than the state-of-the-art methods in SER.

Entropy of Neuronal Spike Patterns.

Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content of neuronal patterns, entropy measures provide insights into neural coding strategies, synaptic plasticity, network dynamics, and cognitive processes. Here, we review basic entropy metrics and then we provide examples of recent advancements in using entropy as a tool to improve our understanding of neuronal processing. It focuses especially on studies on critical dynamics in neural networks and the relation of entropy to predictive coding and cortical communication. We highlight the necessity of expanding entropy measures from single neurons to encompass multi-neuronal activity patterns, as cortical circuits communicate through coordinated spatiotemporal activity patterns, called neuronal packets. We discuss how the sequential and partially stereotypical nature of neuronal packets influences the entropy of cortical communication. Stereotypy reduces entropy by enhancing reliability and predictability in neural signaling, while variability within packets increases entropy, allowing for greater information capacity. This balance between stereotypy and variability supports both robustness and flexibility in cortical information processing. We also review challenges in applying entropy to analyze such spatiotemporal neuronal spike patterns, notably, the "curse of dimensionality" in estimating entropy for high-dimensional neuronal data. Finally, we discuss strategies to overcome these challenges, including dimensionality reduction techniques, advanced entropy estimators, sparse coding schemes, and the integration of machine learning approaches. Thus, this work summarizes the most recent developments on how entropy measures contribute to our understanding of principles underlying neural coding.

A new spectral measure of complexity and its capabilities for detecting signals in noise

This article is devoted to the improvement of signal recognition methods based on the information characteristics of the spectrum. A discrete function of the normalized ordered spectrum is established for a single window function included in the DFT. Lemmas on estimates of entropy, imbalance and statistical complexity in processing a time series of independent Gaussian quantities are proved. New concepts of one-dimensional and two-dimensional spectral complexities are proposed. The theoretical results obtained were verified by numerical experiments, which confirmed the effectiveness of the new information characteristic when detecting a signal mixed with white noise at low signal-to-noise ratios.

The use of Approximate Entropy analysis for flow pattern identification in radial compressors to detect instable operating conditions

Abstract This study uses a statistical analysis on given pressure signals to identify the surge condition in radial compressors, providing insight into their dynamics and performance. Specifically, the investigation focuses on a two-stage centrifugal compressor used in refrigeration applications. Through Computational Fluid Dynamics (CFD) Unsteady Reynolds-Averaged Navier-Stokes (URANS) simulations, static pressure time-series are collected to cover four distinct operational points of the machine, ranging from a stable condition to an unstable one. To assess the stability of the compressor operation, the analysis introduces the Approximate Entropy (ApEn) as a statistical parameter to measure the regularity and predictability of the time-series signals. The results show that the proposed methodology is effective in pattern recognition, able to identify sources of instabilities, and to distinguish between different operating conditions. This approach enhances the comprehension of the operational characteristics of centrifugal compressors and sets the stage for improved monitoring and diagnostic methodologies. ApEn can therefore be a useful index to analyse time-series from sensors easy to install in real compressor configurations. The proposed methodology is included in a set of models, based on machine learning techniques, previously developed by the authors, to identify compressor instability during operation.

Implementation of entropically secure encryption: Securing personal health data

AbstractEntropically secure encryption (ESE) offers unconditional security with shorter keys compared to the One‐Time Pad. Here, the first implementation of ESE for bulk encryption is presented. The main computational bottleneck for bulk ESE is a multiplication in a very large finite field. This involves multiplication of polynomials followed by modular reduction. A polynomial multiplication is implemented based on the gf2x library, with modifications that avoid inputs of vastly different length, thus improving speed. Additionally, a recently proposed efficient reduction algorithm that works for any polynomial degree is implemented. Two use cases are investigated: x‐ray images of patients and human genome data. Entropy estimation is conducted using compression methods whose results determine the key lengths required for ESE. The running times for all steps of the encryption are reported. The potential of ESE to be used in conjunction with quantum key distribution (QKD), in order to achieve full information‐theoretic security of QKD‐protected links for these use cases is discussed.

Density estimation for ordinal biological sequences and its applications.

Biological sequences do not come at random. Instead, they appear with particular frequencies that reflect properties of the associated system or phenomenon. Knowing how biological sequences are distributed in sequence space is thus a natural first step toward understanding the underlying mechanisms. Here we propose a method for inferring the probability distribution from which a sample of biological sequences were drawn for the case where the sequences are composed of elements that admit a natural ordering. Our method is based on Bayesian field theory, a physics-based machine learning approach, and can be regarded as a nonparametric extension of the traditional maximum entropy estimate. As an example, we use it to analyze the aneuploidy data pertaining to gliomas from The Cancer Genome Atlas project. In addition, we demonstrate two follow-up analyses that can be performed with the resulting probability distribution. One of them is to investigate the associations among the sequence sites. This provides a way to infer the governing biological grammar. The other is to study the global geometry of the probability landscape, which allows us to look at the problem from an evolutionary point of view. It can be seen that this methodology enables us to learn from a sample of sequences about how a biological system or phenomenon in the real world works.

Methods for faster estimation of the entropy profile of a lithium-ion battery: A comparison of accelerated potentiometry and the estimation of entropy through thermal signatures

The operating temperature of a lithium-ion battery (LIB) has strong effects on its electrochemical performance, safety, and lifetime. Reliable predictions of cell temperature and heat generation rate are required for the design and thermal management of battery systems. The entropy variation of a LIB has a significant influence on the heat generation and the cell temperature variation, especially at a lower C-rate. Since the entropy is a function of State-of-Charge (SoC), and may change during ageing, the entropy profile may as well be used as an indicator for battery State-of-Health diagnostics. The present experimental methods, such as potentiometric or calorimetric methods, for determining the entropy variation throughout a battery's full SoC window are time-consuming and require specialized equipment with good thermal insulation and instrumentation. Hence, faster and more convenient methods are needed. In this paper, we compare an accelerated potentiometric method with a new entropy characterization method where the entropy profile is extracted from the thermal signature of a LIB during charge and discharge at low C-rates. The accelerated potentiometric method aims to obtain the entropy from thermal cycling while the cell is still not completely relaxed to its stable OCV. The thermal signature method monitors the variations of a cell's outer surface temperature during a low constant current charge or discharge, from which the continuous variation of entropy with SoC can be extracted. The obtained entropy profile was validated through a modeling approach that combines a P2D-electrochemical model for battery operation and a 3D-finite-element model for the thermal behavior during battery operation. Good agreement between the model predictions and experiments was achieved on the temporal variation of the cell's outer surface temperature.

High efficiency deep image compression via channel-wise scale adaptive latent representation learning

Recent learning based neural image compression methods have achieved impressive rate–distortion (RD) performance via the sophisticated context entropy model, which performs well in capturing the spatial correlations of latent features. However, due to the dependency on the adjacent or distant decoded features, existing methods require an inefficient serial processing structure, which significantly limits its practicability. Instead of pursuing computationally expensive entropy estimation, we propose to reduce the spatial redundancy via the channel-wise scale adaptive latent representation learning, whose entropy coding is spatially context-free and parallelizable. Specifically, the proposed encoder adaptively determines the scale of the latent features via a learnable binary mask, which is optimized with the RD cost. In this way, lower-scale latent representation will be allocated to the channels with higher spatial redundancy, which consumes fewer bits and vice versa. The downscaled latent features could be well recovered with a lightweight inter-channel upconversion module in the decoder. To compensate for the entropy estimation performance degradation, we further develop an inter-scale hyperprior entropy model, which supports the high efficiency parallel encoding/decoding within each scale of the latent features. Extensive experiments are conducted to illustrate the efficacy of the proposed method. Our method achieves bitrate savings of 18.23%, 19.36%, and 27.04% over HEVC Intra, along with decoding speeds that are 46 times, 48 times, and 51 times faster than the baseline method on the Kodak, Tecnick, and CLIC datasets, respectively.

Shannon Wavelet‐Based Approximation Scheme for Information Entropy Integrals in Confined Domain

ABSTRACTIn this work, the author attempted to develop a Shannon wavelet‐based numerical scheme to approximate the information entropies in both configuration and momentum space corresponding to the ground and adjacent excited energy states of one‐dimensional Schrödinger equation appearing in non‐relativistic quantum mechanics. The development of this scheme is based on the judicious use of sinc scale functions as an approximation basis and a suitable numerical quadrature to approximate entropies in position and momentum spaces. Priori and posteriori errors appearing in the approximations of wave functions and entropy integrals have been discussed. The scheme (coded in Python) has been subsequently exercised for various exactly solvable and quasi‐exactly solvable non‐relativistic quantum mechanical models in confined domain.

Entropy-Based Volatility Analysis of Financial Log-Returns Using Gaussian Mixture Models.

Volatility in financial markets refers to the variation in asset prices over time. High volatility indicates increased risk, making its evaluation essential for effective risk management. Various methods are used to assess volatility, with the standard deviation of log-returns being a common approach. However, this implicitly assumes that log-returns follow a Gaussian distribution, which is not always valid. In this paper, we explore the use of (differential) entropy to evaluate the volatility of financial log-returns. Estimation of entropy is obtained using a Gaussian mixture model to approximate the underlying density of log-returns. Following this modeling approach, popular risk measures such as Value at Risk and Expected Shortfall can also be computed. By integrating Gaussian mixture modeling and entropy into the analysis of log-returns, we aim to provide a more accurate and robust framework for assessing financial volatility and risk measures.

Heart rate variability and insomnia in depressed patients with breast cancer

ObjectivesDepression is associated with unhealthy autonomic regulation. However, whether patients with breast cancer (BC) with different degrees of depression can be identified from linear and non-linear dynamics in the autonomic...

Relaxed Equilibria for Time-Inconsistent Markov Decision Processes

This paper considers an infinite-horizon Markov decision process (MDP) that allows for general nonexponential discount functions in both discrete and continuous time. Because of the inherent time inconsistency, we look for a randomized equilibrium policy (i.e., relaxed equilibrium) in an intrapersonal game between an agent’s current and future selves. When we modify the MDP by entropy regularization, a relaxed equilibrium is shown to exist by a nontrivial entropy estimate. As the degree of regularization diminishes, the entropy-regularized MDPs approximate the original MDP, which gives the general existence of a relaxed equilibrium in the limit by weak convergence arguments. As opposed to prior studies that consider only deterministic policies, our existence of an equilibrium does not require any convexity (or concavity) of the controlled transition probabilities and reward function. Interestingly, this benefit of considering randomized policies is unique to the time-inconsistent case.

Quantifying Hand Motion Complexity in Simulated Sailing Using Inertial Sensors.

The control of hand movement during sailing is important for performance. To quantify the amount of regularity and the unpredictability of hand fluctuations during the task, the mathematical algorithm Approximate Entropy (ApEn) of the hand acceleration can be used. Approximate Entropy is a mathematical algorithm that depends on the combination of two input parameters including (1) the length of the sequences to be compared (m), and (2) the tolerance threshold for accepting similar patterns between two segments (r). The aim of this study is to identify the proper combinations of 'm' and 'r' parameter values for ApEn measurement in the hand movement acceleration data during sailing. Inertial Measurement Units (IMUs) recorded acceleration data for both the mainsail (non-dominant) and tiller (dominant) hands across the X-, Y-, and Z-axes, as well as vector magnitude. ApEn values were computed for 24 parameter combinations, with 'm' ranging from 2 to 5 and 'r' from 0.10 to 0.50. The analysis revealed significant differences in acceleration ApEn regularity between the two hands, particularly along the Z-axis, where the mainsail hand exhibited higher entropy values (p = 0.000673), indicating greater acceleration complexity and unpredictability. In contrast, the tiller hand displayed more stable and predictable acceleration patterns, with lower ApEn values. ANOVA results confirmed that parameter 'm' had a significant effect on acceleration complexity for both hands, highlighting differing motor control demands between the mainsail and tiller hands. These findings demonstrate the utility of IMU sensors and ApEn in detecting nuanced variations in acceleration dynamics during sailing tasks. This research contributes to the understanding of hand-specific acceleration patterns in sailing and provides a foundation for further studies on adaptive sailing techniques and motor control strategies for both novice and expert sailors.

Enhancing gait recognition in lower limb exoskeletons: adaptive feature selection and random forest with Bayesian optimization

Abstract To improve human-machine cooperation and enhance the accuracy of gait recognition in wearable lower limb exoskeletons, an enhancement method of gait recognition based on adaptive feature selection and an optimized machine learning algorithm was proposed. In this study, surface electromyography (sEMG) signals of rectus femoris, medialis femoris, lateralis femoris, semitendinosus and biceps femoris were recorded during level-ground walking. Then, time domain, frequency domain, time-frequency domain and nonlinear features were extracted. The integrated value of electromyography, variance, root mean square and wavelength were selected as the time domain features, and the mean power frequency and median frequency were selected as the frequency domain features. Wavelet packet energy was selected as the time-frequency domain feature. Nonlinear features including approximate entropy, sample entropy and fuzzy entropy of sEMG were extracted. To reduce feature dimension and improve the calculation efficiency, adaptive feature selection was performed by particle swarm optimization combined with sigmoid function. Then, the feature matrix was determined as the input for a random forest classifier to recognize different gait phases. An adaptive optimization mechanism based on Bayesian optimization was applied to random forest. Compared with random forest, the overall performance of the optimized model was improved. Its accuracy was increased by 3.57%. The feature selection and adaptive optimization mechanisms in gait recognition not only enhance the accuracy of random forest algorithms applied to sEMG for gait prediction, but also facilitate the flexibility and consistency required for lower limb exoskeleton gait control.

Data-driven system identification and model predictive control of pneumatic conveying using nonlinear dynamics analysis for optimised energy consumption

Pneumatic conveying systems provide secure transportation of particulate material and a dust-free environment. These systems face high energy consumption, material degradation, and pipeline blockages. This research presents an innovative solution by integrating nonlinear dynamics analysis of electrostatic sensor data, including chaos and recurrence quantification analysis, sparse identification of nonlinear dynamics with control (SINDYc) and model predictive control (MPC). The Lyapunov exponent, approximate entropy and recurrence rate of electrostatic sensor data reveal the chaotic nature of gas-solid flows. MPC framework was tailored for real-time optimisation of a pneumatic conveying system. SINDYc system models were developed using data collected from an open-loop control pneumatic conveying process to conduct MPC simulations and select an appropriate model for real-time control. This research illustrates the potential of integrating nonlinear dynamics analysis, SINDYc integrated and MPC for enhanced system performance, showcased a significant reduction in energy consumption without compromising the system's efficiency or reliability.

CONTINUOUS SCORING OF AUTISM SPECTRUM DISORDER PATIENTS BY ANALYZING THEIR EEG SIGNALS

Clinical manifestations and standard psychological tests have been widely used to diagnose autism spectrum disorder (ASD) patients and evaluate their severity level. The gold-standard criterion to diagnose ASD patients is the childhood autism rating scale (CARS), which is a qualitative questionnaire that is filled out through a systematic interview while no physiological test/record is performed to determine this score. To make the diagnosis process quantitative, electroencephalography (EEG) signals have been repeatedly analyzed to differentiate healthy subjects from ASD patients. However, the precise relationship between the abnormal behavior of EEG signals and different ASD severity levels is not well investigated. Here, we use CARS to qualitatively determine the severity level of 14 autistic children, who voluntarily enrolled in our study. We recorded their EEG signals from 19 scalp channels when they were awake in the idle state and elicited three informative features including approximation entropy, multiscale entropy and sample entropy in successive time frames. Among the three measures of entropy, the last one exhibits the highest sensitivity, where its correlation coefficient (CC) exceeds 0.7, on the electrode positions T6 (CC [Formula: see text] 0.74), P4 (CC [Formula: see text] 0.76) and Cz (CC [Formula: see text] 0.75). Results of sample entropy in channels Cz-versus-Pz and P4-versus-FP2 show that a simple K-nearest neighbor classifier can provide 93% classification accuracy among patients with mild, moderate and severe ASD levels. Comparing the proposed method to the conventional ones, we also extracted power spectral density features from the channels but they failed to identify the ASD severity level with an acceptable accuracy.

Key Rank Estimation Methods: Comparisons and Practical Considerations

New proposals for scalable key rank estimation methods have appeared recently, in particular the sampling based approach MCRank. The idea is that one can consistently estimate the key rank by sampling only a small portion of the key space as a “proxy”, leading to both an accurate and scalable approach, at least in comparison with another approach based on histograms. We show that the (earlier) GEEA algorithm is in fact a sampling based algorithm, and provide an in-depth comparison between GEEA (when adapted to produce rank estimates rather than guessing entropy estimates), GM bounds, MCRank and the currently most performant counting based rank estimation as implemented in the Labynkyr library. We find that although MCRank does live up to the promised accuracy and scalability for probability-based distinguishers, it fails to handle cases with unusual distinguisher distributions. Furthermore, we put forward a novel proposal for a highly scalable key rank estimation method by introducing the notion of an “attacker budget”. Our proposal is based on the idea that, in particular for very long keys, the exact key rank is less important than the knowledge whether a key is within a certain bound. Thus our “budget approach” is based on efficiently checking if the result of an attack is such that the attacker's budget suffices for successful enumeration. Our budget approach scales linearly with the key size and thus enables security estimations even for post-quantum key lengths.