Definition Of Entropy Research Articles

Relative State Counting for Semiclassical Black Holes

It has been shown that entropy differences between certain states of perturbative quantum gravity can be computed without specifying an ultraviolet completion. This is analogous to the situation in classical statistical mechanics, where entropy differences are defined but absolute entropy is not. Unlike in classical statistical mechanics, however, the entropy differences computed in perturbative quantum gravity do not have a clear physical interpretation. Here we construct a family of perturbative black hole states for which the entropy difference can be interpreted as a relative counting of states. Conceptually, this Letter begins with the algebra of mass fluctuations around a fixed black hole background, and points out that while this is a type I algebra, it is not a factor and therefore has no canonical definition of entropy. As in previous work, coupling the mass fluctuations to quantum matter embeds the mass algebra within a type II factor, in which entropy differences (but not absolute entropies) are well defined. It is then shown that for microcanonical wave functions of mass fluctuation, the type II entropy difference equals the logarithm of the dimension of the extra Hilbert space that is needed to map one microcanonical window to another using gauge-invariant unitaries. The Letter closes with comments on type II entropy difference in a more general class of states, where the von Neumann entropy difference does not have a physical interpretation, but “one-shot” entropy differences do. Published by the American Physical Society 2024

A Heuristic Attribute-Reduction Algorithm Based on Conditional Entropy for Incomplete Information Systems

With the continuous expansion of databases, the extraction of information has been an urgent research topic in many fields. As an effective method to remove redundant attributes, attribute reduction demonstrates extraordinary ability in simplifying information systems. This paper applies a novel form of conditional entropy to investigate the attribute reduction in incomplete information systems. Firstly, a novel definition of conditional entropy is introduced based on tolerance relation. Additionally, in order to reduce time complexity, we propose a binsearch heuristic attribute-reduction algorithm with conditional entropy as heuristic knowledge. Furthermore, two examples are used to illustrate the feasibility and validity of the reduction algorithm.

The Representative Points of Generalized Alpha Skew-t Distribution and Applications

Assuming the underlying statistical distribution of data is critical in information theory, as it impacts the accuracy and efficiency of communication and the definition of entropy. The real-world data are widely assumed to follow the normal distribution. To better comprehend the skewness of the data, many models more flexible than the normal distribution have been proposed, such as the generalized alpha skew-t (GAST) distribution. This paper studies some properties of the GAST distribution, including the calculation of the moments, and the relationship between the number of peaks and the GAST parameters with some proofs. For complex probability distributions, representative points (RPs) are useful due to the convenience of manipulation, computation and analysis. The relative entropy of two probability distributions could have been a good criterion for the purpose of generating RPs of a specific distribution but is not popularly used due to computational complexity. Hence, this paper only provides three ways to obtain RPs of the GAST distribution, Monte Carlo (MC), quasi-Monte Carlo (QMC), and mean square error (MSE). The three types of RPs are utilized in estimating moments and densities of the GAST distribution with known and unknown parameters. The MSE representative points perform the best among all case studies. For unknown parameter cases, a revised maximum likelihood estimation (MLE) method of parameter estimation is compared with the plain MLE method. It indicates that the revised MLE method is suitable for the GAST distribution having a unimodal or unobvious bimodal pattern. This paper includes two real-data applications in which the GAST model appears adaptable to various types of data.

Combating Misinformation on Social Media Using Social Noise and Social Entropy as a Measure of Uncertainty

ABSTRACTRecent events around the world, including the war in Ukraine and the conflict in Gaza have highlighted the effective use of social media as a tool to voice concerns about social issues to create awareness. At the same time, social media has become a fertile ground for spreading misinformation, fake news, and conspiracy theories. Misinformation and conspiracy theories have existed since the existence of mankind. What is new today is the speed by which misinformation can be created, magnified and spread using social media. Efforts to regulate social media and control the widespread spread of misinformation are still lacking due to rapid advances in technology and concerns regarding free speech. One approach to minimizing the impact of misinformation is to focus on social noise as an important factor in magnifying and spreading misinformation. In this paper, we investigate methods of identifying and quantifying social noise using social entropy as a measure of uncertainty and topic modeling. Results from the study have shown a direct relationship between social noise and social entropy. The results have also shown that social noise and social entropy decrease with the use of URLs and rich content (sematic information). Further studies will include the use of machine learning and AI techniques to improve the definition of social news and social entropy.

Complexity Measures in the Tight-Binding Model

Abstract The deformation of a wave packet is a significant topic in classical and quantum mechanics. Understanding this phenomenon is relevant in the study of various physical systems. In this work, we characterize the evolution of a highly localized wave packet in a tight-binding lattice. We investigate the behavior of the probability distribution associated with the wave packet and the accompanying complexity measures. We take information entropy, disequilibrium, disorder, and complexity measures to describe the localization-delocalization process from a highly localized initial pulse, showing the particles moving in a lattice. The main result is obtained from the entropy definition (Logarithmic and Linear) and the inverse of the participant ratio to describe the expected localization-delocalization process, evoking two definitions of Complexity: CLMC and CSDL .

DeltaGzip: Computing Biopolymer-Ligand Binding Affinity via Kolmogorov Complexity and Lossless Compression.

The design of biosequences for biosensing and therapeutics is a challenging multistep search and optimization task. In principle, computational modeling may speed up the design process by virtual screening of sequences based on their binding affinities to target molecules. However, in practice, existing machine-learned models trained to predict binding affinities lack the flexibility with respect to reaction conditions, and molecular dynamics simulations that can incorporate reaction conditions suffer from high computational costs. Here, we describe a computational approach called DeltaGzip that evaluates the free energy of binding in biopolymer-ligand complexes from ultrashort equilibrium molecular dynamics simulations. The entropy of binding is evaluated using the Kolmogorov complexity definition of entropy and approximated using a lossless compression algorithm, Gzip. We benchmark the method on a well-studied data set of protein-ligand complexes comparing the predictions of DeltaGzip to the free energies of binding obtained using Jarzynski equality and experimental measurements.

EVALUATION OF WOOD DAMAGE AND FRACTURE BEHAVIOR BASED ON ENERGY ENTROPY OF ACOUSTIC EMISSION SIGNALS

In order to assess the damage and fracture behavior of wood under continuous loading, an energy entropy and b-value associated with the acoustic emission (AE) signal were defined to quantitatively describe the release of strain energy during loading. Firstly, the acoustic emission signals of the wood in the three-point bending test were collected. This paper presents the concept of energy entropy according to the definition of information entropy. In order to further evaluate the strain energy intensity released by the damage behavior of the wood specimen, the acoustic emission b-value was defined. Finally, by jointly analysing the dynamics of these two parameters, the test process can be divided into three phases. The results show that even in the elastic phase, micro-destructive behavior occur inside the wood specimen; in the plastic phase, the wood specimen is not only subjected to macroscopic damage, but also often accompanied by fine cracks inside.

EDMD: An Entropy based Dissimilarity measure to cluster Mixed-categorical Data

The effectiveness of clustering techniques is significantly influenced by proximity measures irrespective of type of data and categorical data is no exception. Most of the existing proximity measures for categorical data assume that all attributes contribute equally to the distance measurement which is not true. Usually, frequency or probability-based approaches are better equipped in principle to counter this issue by appropriately weighting the attributes based on the intra-attribute statistical information. However, owing to the qualitative nature of categorical features, the intra-attribute disorder is not captured effectively by the popularly used continuum form of entropy known as Shannon or information entropy. If the categorical data contains ordinal features, then the problem multiplies because the existing measures treat all attributes as nominal. To address these issues, we propose a new Entropy-based Dissimilarity measure for Mixed categorical Data (EDMD) composed of both nominal and ordinal attributes. EDMD treats both nominal and ordinal attributes separately to capture the intrinsic information from the values of two different attribute types. We apply Boltzmann’s definition of entropy, which is based on the principle of counting microstates, to exploit the intra-attribute statistical information of nominal attributes while preserving the order relationships among ordinal values in distance formulation. Additionally, the statistical significance of different attributes of the data towards dissimilarity computation is taken care of through attribute weighting. The proposed measure is free from any user-defined or domain-specific parameters and there is no prior assumption about the distribution of the data sets. Experimental results demonstrate the efficacy of EDMD in terms of cluster quality, accuracy, cluster discrimination ability, and execution time to handle mixed categorical data sets of different characteristics.

Differential Shannon Entropies and Mutual Information in a Curve Crossing System: Adiabatic and Diabatic Decompositions.

The differential Shannon entropy provides a measure for the localization of a wave function. We regard the vibrational wave packet motion in a curve crossing system and calculate time-dependent entropies. Using a numerical example, we analyze how localization inside diabatic and adiabatic states can be accessed and discuss the differences between these two representations. In order to do so, we extend the usual entropy definition and introduce novel state-selective entropies. These quantities contain information on the form of the nuclear density components on the one hand and on the state population on the other, and it is shown how the contribution of the population can be removed. Having the state-selective entropies at hand, two additional functions derived from these, namely, the conditional entropy and the mutual information, are determined and compared. We find that these quantities relate closely to correlation effects rooted in different electronic properties of the system.

Conditional plausibility entropy of belief functions based on Dempster conditioning

Uncertainty quantification of belief functions is an unsolved issue in belief function theory that is used widely to tackle epistemic uncertainty. This study provides a new definition of conditional entropy of belief functions, called conditional plausibility entropy, in terms of Dempster conditioning and plausibility entropy of mass functions. The proposed conditional plausibility entropy mainly meets two aspects of good properties, one is the semantics of independence property, the other is the compatibility with Shannon's conditional entropy for probability distributions, which are not simultaneously satisfied by existing definitions of conditional entropy of belief functions. The effectiveness of conditional plausibility entropy is validated further through comparison and analysis on the basis of different conditioning methods.

Parametric topological entropy on orbits of arbitrary multivalued maps in compact Hausdorff spaces

The Adler-Konheim-McAndrew type definitions and the Bowen-Dinaburg-Hood type definitions of parametric topological entropy will be considered on orbits and coincidence orbits of nonautonomous multivalued maps in compact Hausdorff spaces. Their mutual relationship and their link to various further types of definitions like those of (parametric) preimage entropy, will be investigated. In this way, several recent results of the other authors will be generalized and extended into a new setting.

Probabilistic and maximum entropy modeling of chemical reaction systems: Characteristics and comparisons to mass action kinetic models.

We demonstrate and characterize a first-principles approach to modeling the mass action dynamics of metabolism. Starting from a basic definition of entropy expressed as a multinomial probability density using Boltzmann probabilities with standard chemical potentials, we derive and compare the free energy dissipation and the entropy production rates. We express the relation between entropy production and the chemical master equation for modeling metabolism, which unifies chemical kinetics and chemical thermodynamics. Because prediction uncertainty with respect to parameter variability is frequently a concern with mass action models utilizing rate constants, we compare and contrast the maximum entropy model, which has its own set of rate parameters, to a population of standard mass action models in which the rate constants are randomly chosen. We show that a maximum entropy model is characterized by a high probability of free energy dissipation rate and likewise entropy production rate, relative to other models. We then characterize the variability of the maximum entropy model predictions with respect to uncertainties in parameters (standard free energies of formation) and with respect to ionic strengths typically found in a cell.

Positive-Incentive Noise.

Noise is conventionally viewed as a severe problem in diverse fields, e.g., engineering and learning systems. However, this brief aims to investigate whether the conventional proposition always holds. It begins with the definition of task entropy, which extends from the information entropy and measures the complexity of the task. After introducing the task entropy, the noise can be classified into two kinds, positive-incentive noise (Pi-noise or π -noise) and pure noise, according to whether the noise can reduce the complexity of the task. Interestingly, as shown theoretically and empirically, even the simple random noise can be the π -noise that simplifies the task. π -noise offers new explanations for some models and provides a new principle for some fields, such as multitask learning, adversarial training, and so on. Moreover, it reminds us to rethink the investigation of noises.

Coarse-graining black holes out of equilibrium with boundary observables on time slice

In black hole thermodynamics, defining coarse-grained entropy for dynamical black holes has long been a challenge, and various proposals, such as generalized entropy, have been explored. Guided by the AdS/CFT, we introduce a new definition of coarse-grained entropy for a dynamical black hole in Lorentzian Einstein gravity. On each time slice, this entropy is defined as the horizon area of an auxiliary Euclidean black hole that shares the same mass, (angular) momenta, and asymptotic normalizable matter modes with the original Lorentzian solution. The entropy is shown to satisfy a generalized first law and, through holography, the second law as well. Furthermore, by applying this thermodynamics to several Vaidya models in AdS and flat spacetime, we discover a connection between the second law and the null energy condition.

Quantum Mechanics Based on an Extended Least Action Principle and Information Metrics of Vacuum Fluctuations

We show that the formulations of non-relativistic quantum mechanics can be derived from an extended least action principle. The principle can be considered as an extension of the least action principle from classical mechanics by factoring in two assumptions. First, the Planck constant defines the minimal amount of action a physical system needs to exhibit during its dynamics in order to be observable. Second, there is constant vacuum fluctuation along a classical trajectory. A novel method is introduced to define the information metrics to measure additional observability due to vacuum fluctuations, which is then converted to an additional action through the first assumption. Applying the variational principle to minimize the total actions allows us to recover the basic quantum formulations including the uncertainty relation and the Schrödinger equation in the position representation. In the momentum representation, the same method can be applied to obtain the Schrödinger equation for a free particle while further investigation is still needed for a particle with an external potential. Furthermore, the principle brings in new results on two fronts. At the conceptual level, we find that the information metrics for vacuum fluctuations are responsible for the origin of the Bohm quantum potential. Even though the Bohm potential for a bipartite system is inseparable, the underlying vacuum fluctuations are local. Thus, inseparability of the Bohm potential does not justify a non-local causal relation between the two subsystems. At the mathematical level, quantifying the information metrics for vacuum fluctuations using more general definitions of relative entropy results in a generalized Schrödinger equation that depends on the order of relative entropy. The extended least action principle is a new mathematical tool. It can be applied to derive other quantum formalisms such as quantum scalar field theory.

Generalized Shannon entropy sparse wavelet packet transform for fault detection of traction motor bearings in high-speed trains

An effective structural health monitoring method of traction motor bearings is a powerful guarantee for the safety operation of high-speed trains. However, it is exceptionally difficult to detect bearing fault characteristics from the vibration signals of traction motor bearings operating at high rotational speeds. In this scenario, a generalized Shannon entropy sparse wavelet packet transform (GSWPT) for fault detection of motor bearings is proposed in this paper. Firstly, a generalized Shannon entropy sparse regularization method is proposed to obtain sparse wavelet reconstruction coefficients by extending the definition of the Shannon information entropy, and the non-convex sparse regularization function is minimized by synergistic swarm optimization algorithm. Then, the wavelet node coefficients are weighted according to the second-order cyclostationarity index of the wavelet packet node to further enhance the sparsity of the reconstructed signal. Moreover, the optimal decomposition level of GSWPT is adaptively selected by the maximum sparsity and cyclostationarity criterion. Particularly, in order to verify the bearing fault detection performance of GSWPT in practical engineering, a bearing fault dynamic model of traction motor in high-speed train was established based on Hertz contact theory and the fourth-order Runge-Kutta method to obtain simulated data under strong Gaussian white noise, and a corresponding test platform was constructed to collect experimental data under different operating conditions. Finally, the applications on the simulated and experimental signals of traction motor bearings in high-speed trains demonstrate that GSWPT significantly outperforms the conventional wavelet packet transform, dual-tree complex wavelet packet transform, blind deconvolution, modal decomposition, and Infogram methods to some extent for fault detection.

Graph-based minimum error entropy Kalman filtering

When the Gaussian kernel function is chosen with a small kernel bandwidths, the minimum error entropy Kalman filter (MEEKF) exhibits excellent performance. However, further narrowing the kernel bandwidths leads to a degradation in performance. To address this issue, this paper proposed a Graph-based MEEKF (G-MEEKF) algorithm based on Graph Signal Processing (GSP) theory. The G-MEEKF algorithm addresses the problem by smoothing the errors using graph filtering and incorporating graph topology into the cost function. The convergence of the algorithm is theoretically analyzed and simulations demonstrate that G-MEEKF improves the performance of MEEKF under small kernel bandwidths. Furthermore, it is emphasized that the topology of the graph abstracted from the minimum error entropy (MEE) definition also influences the performance of the algorithm.

Space-time entropy, space of singularities and gravity origin: A case study

A new definition of entropy is introduced using a model that simulates an expanding space-time compatible with the fundamental principle of cosmology. The entropy is obtained by mean of a state function that measures the variation of the space-time normal curvature, from a highly compressed space to a lower compressed space. The defined entropy leads to work out a new understanding of the earliest conditions that last for a period estimated to 380,000 years after the Big Bang. It leads to understand via a short period of inflation the process that generates the uniform distribution of matter and energy at the surface of the last scattering. It involves gravitational singularities in a process of gradual decompression propitious to the incubation of matter recombination, and it allows to trace back gravity origin.

A mathematical introduction to complex dynamical systems

This work proposes an investigation of the theory of dynamic systems, focusing on the analysis of entropy in its classical and topological manifestations. Initially, the fundamental concepts of dynamical systems theory are presented, with an emphasis on topological dynamical systems (TDS). Next, definitions of discrete topological entropy and its relationship to dynamical systems are discussed, followed by the introduction of topological entropy pressure as a weighted form of topological entropy. Subsequently, some applications of topological entropy in dynamic systems are examined, highlighting its role in the analysis of chaotic systems and in ergodic theory. Furthermore, we present a new theory, called Topological Ergodic Entropy Theory (TEET), which offers an innovative approach to the analysis of ergodic dynamical systems. Finally, the Ergodic Theory of Turbulent Flow (ETTF) is introduced, exploring the relationship between topological entropy and the ergodic properties of dynamic systems governed by the Navier-Stokes equations. The results presented contribute to a deeper understanding of the complexity of dynamical systems and their applications in various areas of mathematics and physics. The investigation of topological entropy and its applications in dynamical systems offers new insights into the chaotic and stochastic behavior of these systems, while the introduction of new theories, such as ETTF, provides a new perspective for the analysis and modeling of turbulent phenomena.

Tracing the classical molecular nature of entropy generation

The second law of thermodynamics and the concept of positive entropy generation are analyzed following classical statistical mechanics methods. Previously, using the generalized Boltzmann–Gibbs entropy and its associated general entropy conservation relation, positive entropy generation expressions were obtained in agreement with phenomenological results and the work of Boltzmann and Gibbs. In this study, using the general approach, we formally and explicitly trace the specific entropy generation expressions to truncations in the full N-body description of the entropy state to a lower s-body description. Using higher-order superposition approximations, it is formally shown that the generalized Boltzmann–Gibbs entropy in the s-order state is always less than the corresponding Boltzmann–Gibbs entropy in the lower (s − 1)-order state. Using the general form of the entropy conservation equation, entropy generation is shown to be a required compensatory effect to ensure that all physical variables and physical processes associated with heat, work, temperature, etc., are independent of the particular entropy definition state.