Renyi Entropy Research Articles

Enhancing machine learning multi-class fault detection in electric motors through entropy-based analysis

Abstract In the field of electric motor maintenance, this study introduces a transformative approach by inte-grating entropy-based algorithms with machine learning for enhanced multi-class fault detection. Employing Shannon, Renyi, and Tsallis entropy algorithms on standard fault detection measure-ments, the research significantly advances predictive maintenance strategies through a robust, ear-ly-indication, system-agnostic analysis. Detailed examination is conducted, comparing results de-rived from datasets that include statistical features (excluding entropy) against the proposed entro-py-based datasets, when applied to a Multi-Layer Perceptron Classifier. Optimization of the MLPC and all compared algorithms’ hyperparameters is done using the state-of-the-art Optuna tool to dynamically explore each search space, ensuring that each methodology performs adequately in a timely fashion while allowing for adaptation. The results showcase significant enhancement in clas-sification accuracy of diverse electric motor operational states, facilitating the differentiation be-tween healthy and various levels of fault conditions under assorted load scenarios. Computational analyses reveal favorable results related to execution time and memory overhead, thereby support-ing the practicality in operations constrained by memory resources. Validation of the approach is achieved through laboratory experiments on a purpose-built test bench. Versatility of entropy-based measures through their proposed utilization in diverse fault indications is achieved by a demonstration in the field of mechanical fault detection with a focus on bearing faults through well-respected datasets.

A New Lifetime Distribution and its Application to Cancer Data

Introduction: Recently, researchers have introduced new generated families of univariate lifetime distributions. These new generators are obtained by adding one or more extra shape parameters to the underlying distribution or compounding two distributions to get more flexibility in fitting data in different areas such as medical sciences, environmental sciences, and engineering. The addition of parameter(s) has been proven useful in exploring tail properties and for improving the goodness-of-fit of the family of the proposed distributions. Methods:A new Three-Parameter Weibull-Generalized Gamma distribution which provides more flexibility in modeling lifetime data is developed using a two-component mixture of Weibull distribution (with parameters θ and λ) and Generalised Gamma distribution (with parameters α=4,θ and λ). Some of its mathematical properties such as the density function, cumulative distribution function, survival function, hazard rate function, moment generating function, Renyi entropy and order statistics are obtained. The maximum likelihood estimation method was used in estimating the parameters of the proposed distribution and a simulation study is performed to examine the performance of the maximum likelihood estimators of the parameters. Results: Real life applications of the proposed distribution to two cancer datasets are presented and its fit was compared with the fit attained by some existing lifetime distributions to show how the Three-Parameter Weibull-Generalized Gamma distribution works in practice Conclusion: The results suggest that the proposed model performed better than its competitors and it’s a useful alternative to the existing models.

Logarithmic singularities of Renyi entropy as a sign of chaos?

We propose that the logarithmic singularities of the Renyi entropy of local-operator-excited states for replica index n can be a sign of quantum chaos. As concrete examples, we analyze the logarithmic singularities of the Renyi entropy in various two-dimensional conformal field theories. We show that there are always logarithmic singularities of the Renyi entropy in holographic CFTs, but no such singularities in free and rational CFTs. These singularities of the Renyi entropy are also related to the logarithmic time growth of the Renyi entropy at late times.

Investigations on collision entropy and its variant for machine condition monitoring

Abstract The efficacy of sparsity measures and complexity measures in machine condition monitoring is well acknowledged. However, a comprehensive exploration of their theoretical underpinnings and interrelations remains incomplete. This study endeavors to address this gap by employing collision entropy as an illustrative case and establishing its relationship with sparsity measures. Through mathematical derivations, it is demonstrated that collision entropy exhibits a phenomenon termed as “bilateral reduction,” a characteristic indicative of complexity measures. Motivated by foundational concepts such as Shannon entropy and relative entropy, this paper introduces the concept of variation in collision entropy, which serves as a sparsity measure. Notably, both Shannon entropy and collision entropy, as specific cases of Rényi entropy, manifest the “bilateral reduction” effect and can be transformed into sparsity measures through certain methodologies. These insights contribute significantly to the exploration of the theoretical properties of Rényi entropy and its conversion into a cluster of sparse measures. This research augments the understanding of sparsity measures and complexity measures, offering theoretical insights crucial for their practical deployment in real-world applications.

Network Data Intrusion Detection and Data Feature Extraction of Electromechanical Facilities from Machine Learning

With the rapid development of Internet technology, network security issues have become more complex and changeable. Various intrusion methods threaten the information network environment in the Electromechanical Facility (EF) system. This paper focuses on EF to study the relevant detection methods and data feature extraction in complex network intrusions. Firstly, four common machine learning algorithms are used to calculate the data set. The advantages and disadvantages of each algorithm are analyzed after tuning and comparison. Secondly, a network intrusion detection algorithm is proposed based on Recursive Feature Elimination (RFE) principal component analysis. It uses RFE to reduce the number of features and improve the elimination judgment index to align with the detection requirements of information network datasets. Finally, a fault diagnosis method is proposed based on empirical pattern decomposition and support vector machine under Renyi entropy complexity measurement. This method trains and identifies the Renyi entropy of several basic pattern components obtained by decomposing empirical patterns as feature vectors. The results show that the RFE method judged by random forest removes irrelevant features, and the evaluation index is improved to align with the network dataset’s detection requirements. It reduces the data dimension, reduces the operation time, and improves the accuracy of a few attack types. The comprehensive final detection effect is better than other algorithms. Additionally, the embedded operating system construction method based on the protection mechanism realizes the separate storage of the operating system and key data. Also, it can prevent the network system from being maliciously invaded, ensuring the stability of the instrument operation under harsh working conditions.

Optimal Twirling Depth for Classical Shadows in the Presence of Noise.

The classical shadows protocol is an efficient strategy for estimating properties of an unknown state ρ using a small number of state copies and measurements. In its original form, it involves twirling the state with unitaries from some ensemble and measuring the twirled state in a fixed basis. It was recently shown that for computing local properties, optimal sample complexity (copies of the state required) is remarkably achieved for unitaries drawn from shallow depth circuits composed of local entangling gates, as opposed to purely local (zero depth) or global twirling (infinite depth) ensembles. Here, we consider the sample complexity as a function of the depth of the circuit, in the presence of noise. We find that this noise has important implications for determining the optimal twirling ensemble. Under fairly general conditions, we (i)show that any single-site noise can be accounted for using a depolarizing noise channel with an appropriate damping parameter f, (ii)compute thresholds f_{th} at which optimal twirling reduces to local twirling for Pauli operators, (iii)nth order Renyi entropies (n≥2), and (iv)provide a meaningful upper bound t_{max} on the optimal circuit depth for any finite noise strength f, which applies to observables and entanglement entropy measurements. These thresholds strongly constrain the search for optimal strategies to implement shadow tomography and are easily tailored to the experimental system at hand.

Capacity of entanglement and volume law

We investigate various aspects of capacity of entanglement in certain setups whose entanglement entropy becomes extensive and obeys a volume law. In particular, considering geometric decomposition of the Hilbert space, we study this measure both in the vacuum state of a family of non-local scalar theories and also in the squeezed states of a local scalar theory. We also evaluate field space capacity of entanglement between interacting scalar field theories. We present both analytical and numerical evidences for the volume law scaling of this quantity in different setups and discuss how these results are consistent with the behavior of other entanglement measures including Renyi entropies. Our study reveals some generic properties of the capacity of entanglement and the corresponding reduced density matrix in the specific regimes of the parameter space. Finally, by comparing entanglement entropy and capacity of entanglement, we discuss some implications of our results on the existence of consistent holographic duals for the models in question.

Capacity of entanglement for scalar fields in squeezed states

We study various aspects of capacity of entanglement in the squeezed states of a scalar field theory. This quantity is a quantum informational counterpart of heat capacity and characterizes the width of the eigenvalue spectrum of the reduced density matrix. In particular, we carefully examine the dependence of capacity of entanglement and its universal terms on the squeezing parameter in the specific regimes of the parameter space. Remarkably, we find that the capacity of entanglement obeys a volume law in the large squeezing limit. We discuss how these results are consistent with the behavior of other entanglement measures including entanglement and Renyi entropies. We also comment on the existence of consistent holographic duals for a family of Gaussian states with generic squeezing parameter based on the ratio of entanglement entropy and the capacity of entanglement. Published by the American Physical Society 2024

Separation of overlapping non-stationary signals and compressive sensing-based reconstruction using instantaneous frequency estimation

Compressive sensing uses an incomplete sample set to consider signal reconstruction with sparse basis representations over a specific transform domain. This work examines the difficulties encountered in various signal processing applications, such as data overlap with the target signal in time and frequency domains. The conventional filtering and windowing processes fail to promote better results in recovering the desired signal. The non-stationary signal with time-frequency representations is highly significant for certain valuable applications. An efficient signal separation and reconstruction method based on instantaneous frequency estimate is proposed to improve the non-stationary signal outcome. The proposed research includes speech signal data acquisition, signal component decomposition, instantaneous frequency estimation, overlapped signal separation, and reconstruction. Two datasets, Dacon Overlapped Speech and LJ Speech Dataset, are utilized to generate a new overlapped signal dataset. Initially, the generated speech signals are collected and decomposed into several monocomponents using Synchrosqueezing Wavelet Transform (SynWav), Renyi's quadratic entropy (RQuad), Narrow bandpass filtering (NBand) and single frequency filtering (SFreq). The instantaneous frequencies are estimated using the Hilbert-Huang transform (HilHuT) to generate a time-frequency representation of multi-component signals. The overlapped signals are separated and reconstructed to a desired signal using a Compressive sensing-based Discrete Cosine transform with amended intrinsic chirp separation (CDcT_AIChirS). The proposed model is simulated using PYTHON to examine the performances. In comparison, the mean square error is calculated to be 2.71, the root mean square error is calculated to be 1.64, and the signal to noise ratio is calculated to be 32dB.

Optimized automated detection of diabetic retinopathy severity: integrating improved multithresholding tunicate swarm algorithm and improved hybrid butterfly optimization.

Diabetic retinopathy, a complication of diabetes, damages the retina due to prolonged high blood sugar levels, leading to vision impairment and blindness. Early detection through regular eye exams and proper diabetes management are crucial in preventing vision loss. DR is categorized into five classes based on severity, ranging from no retinopathy to proliferative diabetic retinopathy. This study proposes an automated detection method using fundus images. Image segmentation divides fundus images into homogeneous regions, facilitating feature extraction. Feature selection aims to reduce computational costs and improve classification accuracy by selecting relevant features. The proposed algorithm integrates an Improved Tunicate Swarm Algorithm (ITSA) with Renyi's entropy for enhanced adaptability in the initial and final stages. An Improved Hybrid Butterfly Optimization (IHBO) Algorithm is also introduced for feature selection. The effectiveness of the proposed method is demonstrated using retinal fundus image datasets, achieving promising results in DR severity classification. For the IDRiD dataset, the proposed model achieves a segmentation Dice coefficient of 98.06% and classification accuracy of 98.21%. In contrast, the E-Optha dataset attains a segmentation Dice coefficient of 97.95% and classification accuracy of 99.96%. Experimental results indicate the algorithm's ability to accurately classify DR severity levels, highlighting its potential for early detection and prevention of diabetes-related blindness.

A Generalized Class of Exponentiated Modified Weibull Distribution With Applications

Abstract: In this paper, a new class of five parameter gamma-exponentiated or generalized modified Weibull (GEMW) distribution which includes exponential, Rayleigh, Weibull, modified Weibull, exponentiated Weibull, exponentiated exponential, exponentiated modified Weibull, exponentiated modified exponential, gamma-exponentiated exponential, gamma exponentiated Rayleigh, gamma-modified Weibull, gamma-modified exponential, gamma-Weibull, gamma-Rayleigh and gamma-exponential distributions as special cases is proposed and studied. Mathematical properties of this new class of distributions including moments, mean deviations, Bonferroni and Lorenz curves, distribution of order statistics and Renyi entropy are presented. Maximum likelihood estimation technique is used to estimate the model parameters and applications to real data sets presented in order to illustrate the usefulness of this new class of distributions and its sub-models.

A New Generalization of the Inverse Generalized Weibull Distribution with Different Methods of Estimation and Applications in Medicine and Engineering

Limitations inherent to existing statistical distributions in capturing the complexities of real-world data often necessitate the development of novel models. This paper introduces the new exponential generalized inverse generalized Weibull (NEGIGW) distribution. The NEGIGW distribution boasts significant flexibility with symmetrical and asymmetrical shapes, allowing its hazard rate function to be adapted to many failure patterns observed in various fields such as medicine, biology, and engineering. Some statistical properties of the NEGIGW distribution, such as moments, quantile function, and Renyi entropy, are studied. Three methods are used for parameter estimation, including maximum likelihood, maximum product of spacing, and percentile methods. The performance of the estimation methods is evaluated via Monte Carlo simulations. The NEGIGW distribution excels in its ability to fit real-world data accurately. Five medical and engineering datasets are applied to demonstrate the superior fit of NEGIGW distribution compared to competing models. This compelling evidence suggests that the NEGIGW distribution is promising for lifetime data analysis and reliability assessments across different disciplines.

An efficient adaptive multilevel Renyi entropy thresholding method based on the energy curve with dynamic programming

An efficient adaptive multilevel Renyi entropy thresholding method based on the energy curve with dynamic programming

On a New Transmuted Three-Parameter Lindley Distribution and Its Applications

In this paper, a new transmuted three-parameter Lindley distribution (TTHPLD) is established using the transmutation map method, which includes the Lindley distribution, two-parameter Lindley distribution, transmuted two-parameter Lindley distribution and three-parameter Lindley distribution as special cases. The statistical properties of the TTHPLD model, which are based on moments, order statistics, hazard rate functions, reliability functions, and Renyi entropy, have been studied. Moreover, the maximum likelihood estimators (MLEs) of the TTHPLD are obtained via differential evolution algorithms, and a simulation study is conducted to evaluate the consistency of the MLEs. Finally, the proposed distribution is applied to a real dataset and compared with other well-known models.

Statistical Divergence and Paths Thereof to Socioeconomic Inequality and to Renewal Processes.

This paper establishes a general framework for measuring statistical divergence. Namely, with regard to a pair of random variables that share a common range of values: quantifying the distance of the statistical distribution of one random variable from that of the other. The general framework is then applied to the topics of socioeconomic inequality and renewal processes. The general framework and its applications are shown to yield and to relate to the following: f-divergence, Hellinger divergence, Renyi divergence, and Kullback-Leibler divergence (also known as relative entropy); the Lorenz curve and socioeconomic inequality indices; the Gini index and its generalizations; the divergence of renewal processes from the Poisson process; and the divergence of anomalous relaxation from regular relaxation. Presenting a 'fresh' perspective on statistical divergence, this paper offers its readers a simple and transparent construction of statistical-divergence gauges, as well as novel paths that lead from statistical divergence to the aforementioned topics.

DRPSO:A multi-strategy fusion particle swarm optimization algorithm with a replacement mechanisms for colon cancer pathology image segmentation

Colon adenocarcinoma (COAD) is a type of colon cancers with a high mortality rate. Its early symptoms are not obvious, and its late stage is accompanied by various complications that seriously endanger patients' lives. To assist in the early diagnosis of COAD and improve the detection efficiency of COAD, this paper proposes a multi-level threshold image segmentation (MIS) method based on an enhanced particle swarm algorithm for segmenting COAD images. Firstly, this paper proposes a multi-strategy fusion particle swarm optimization algorithm (DRPSO) with a replacement mechanism. The non-linear inertia weight and sine-cosine learning factors in DRPSO help balance the exploration and exploitation phases of the algorithm. The population reorganization strategy incorporating MGO enhances population diversity and effectively prevents the algorithm from stagnating prematurely. The mutation-based final replacement mechanism enhances the algorithm's ability to escape local optima and helps the algorithm to obtain highly accurate solutions. In addition, comparison experiments on the CEC2020 and CEC2022 test sets show that DRPSO outperforms other state-of-the-art algorithms in terms of convergence accuracy and speed. Secondly, by combining the non-local mean 2D histogram and 2D Renyi entropy, this paper proposes a DRPSO algorithm based MIS method, which is successfully applied to the segments the COAD pathology image problem. The results of segmentation experiments show that the above method obtains relatively higher quality segmented images with superior performance metrics: PSNR = 23.556, SSIM = 0.825, and FSIM = 0.922. In conclusion, the MIS method based on the DRPSO algorithm shows great potential in assisting COAD diagnosis and in pathology image segmentation.

A modified cosmic brane proposal for holographic Renyi entropy

We propose a new formula for computing holographic Renyi entropies in the presence of multiple extremal surfaces. Our proposal is based on computing the wave function in the basis of fixed-area states and assuming a diagonal approximation for the Renyi entropy. For Renyi index n ≥ 1, our proposal agrees with the existing cosmic brane proposal for holographic Renyi entropy. For n < 1, however, our proposal predicts a new phase with leading order (in Newton’s constant G) corrections to the cosmic brane proposal, even far from entanglement phase transitions and when bulk quantum corrections are unimportant. Recast in terms of optimization over fixed-area states, the difference between the two proposals can be understood to come from the order of optimization: for n < 1, the cosmic brane proposal is a minimax prescription whereas our proposal is a maximin prescription. We demonstrate the presence of such leading order corrections using illustrative examples. In particular, our proposal reproduces existing results in the literature for the PSSY model and high-energy eigenstates, providing a universal explanation for previously found leading order corrections to the n < 1 Renyi entropies.

Synchronizing real-time and high-precision LDoS defense of learning model-based in AIoT with programmable data plane, SDN

The availability of SD-AIoT is currently under complicated and serious cyber threats, especially Low-rate Denial-of-Service attacks. However, traditional defense schemes for such attacks with characteristics of high concealability and periodicity suffer from serious challenges with high detection difficulty, low accuracy of detection models, and inefficiency of mitigation approaches. In this paper, one novel cooperative defense scheme against hybrid LDoS attacks is proposed, which consists of a timely-response hardware-based Renyi Entropy edge checkpoint intent detection algorithm, the high-precision detection mechanism based on a hybrid deep learning model, and a Markov-chain-based differential rate-limiting mitigation strategy. The detection algorithm deployed at the edge checkpoint activates a hybrid CNN-RF-based deep learning model after filtering the intent information of the flows to detect which are malicious LDoS flows with high accuracy, where the multi-stage detection scheme not only extracts and learns the hidden features of the flow data, but also has better representation capabilities. Enhanced dynamic threshold-based whitelisting automatically adapts to the real-time state of the network environment to improve mitigation flexibility. Markov chain-based differential rate-limiting mitigation strategy reduces the packet loss error rate to mitigate network attacks promptly and ensures the continuation of network services. The results of several comparative experiments show that the proposed scheme detects LDoS attacks more accurately and mitigates them more effectively than traditional schemes.

A maximum entropy-driven support vector classification model for seismic collapse fragility curves estimation of reinforced concrete frame structures

A novel method called maximum entropy-driven support vector classification (MED-SVC) for estimating the seismic collapse fragility curves of reinforced concrete (RC) frames is proposed in this paper. In comparison to kernel-based learning (KBL) methods, a subset selection process based on the maximum Renyi entropy criterion is embedded into the proposed MED-SVC, which can actively select an informative small-scale subset from a large-scale training dataset. The selected subset can accurately reflect the data distribution of the large-scale training dataset, which can be further used to explicitly establish a high-dimensional feature variable. Due to this, the computational complexity of training the proposed MED-SVC model only depends on the dimension of the feature variable, which can effectively solve the problem related to the low efficacy of training the KBL model in the context of a large-scale training dataset. Meanwhile, the proposed MED-SVC model can also reduce the negative effect of randomness in the subset selection process on predictive performance. The robustness, accuracy, and efficiency of the proposed MED-SVC model are thoroughly validated by comparison with random sampling driven SVC (RSD-SVC), KBL methods, random forest, neural network, and naive bayes based on 33,000 damage data of RC frames under earthquakes. Then, the proposed MED-SVC model is applied to estimate the seismic collapse fragility curve of an RC frame structure outside the aforementioned dataset. The results show that the proposed MED-SVC model can generate seismic collapse fragility curves that agree well with those produced by the finite element method (FEM) and performs more robustly than RSD-SVC.

Magic in generalized Rokhsar-Kivelson wavefunctions

Magic is a property of a quantum state that characterizes its deviation from a stabilizer state, serving as a useful resource for achieving universal quantum computation e.g., within schemes that use Clifford operations. In this work, we study magic, as quantified by the stabilizer Renyi entropy, in a class of models known as generalized Rokhsar-Kivelson systems, i.e., Hamiltonians that allow a stochastic matrix form (SMF) decomposition. The ground state wavefunctions of these systems can be written explicitly throughout their phase diagram, and their properties can be related to associated classical statistical mechanics problems, thereby allowing powerful analytical and numerical approaches that are not usually available in conventional quantum many body settings. As a result, we are able to express the SRE in terms of wave function coefficients that can be understood as a free energy difference of related classical problems. We apply this insight to a range of quantum many body SMF Hamiltonians, which affords us to study numerically the SRE of large high-dimensional systems, and in some cases to obtain analytical results. We observe that the behaviour of the SRE is relatively featureless across quantum phase transitions in these systems, although it is indeed singular (in its first or higher order derivative, depending on the nature of the transition). On the contrary, we find that the maximum of the SRE generically occurs at a cusp away from the quantum critical point, where the derivative suddenly changes sign. Furthermore, we compare the SRE and the logarithm of overlaps with specific stabilizer states, asymptotically realised in the ground state phase diagrams of these systems. We find that they display strikingly similar behaviors, which in turn establish rigorous bounds on the min-relative entropy of magic.