Definition Of Entropy Research Articles

Accelerating information entropy-based feature selection using rough set theory with classified nested equivalence classes

Feature selection effectively reduces the dimensionality of data. For feature selection, rough set theory offers a systematic theoretical framework based on consistency measures, of which information entropy is one of the most important significance measures of attributes. However, an information-entropy-based significance measure is computationally expensive and requires repeated calculations. Although many accelerating strategies have been proposed thus far, there remains a bottleneck when using an information-entropy-based feature selection algorithm to handle large-scale datasets with high dimensions. In this study, we introduce a classified nested equivalence class (CNEC)-based approach to calculate the information-entropy-based significance for feature selection using rough set theory. The proposed method extracts knowledge of the reducts of a decision table to reduce the universe and construct CNECs. By exploring the properties of different types of CNECs, we can not only accelerate both outer and inner significance calculation by discarding useless CNECs but also effectively decrease the number of inner significance calculations by using one type of CNECs. The use of CNECs is shown to significantly enhance three representative entropy-based feature selection algorithms that use rough set theory. The feature subset selected by the CNEC-based algorithms is the same as that selected by algorithms using the original definition of information entropies. Experiments conducted using 31 datasets from multiple sources, such as the UCI repository and KDD Cup competition, including large-scale and high-dimensional datasets, confirm the efficiency and effectiveness of the proposed method.

Entropy Measures for Plithogenic Sets and Applications in Multi-Attribute Decision Making

Plithogenic set is an extension of the crisp set, fuzzy set, intuitionistic fuzzy set, and neutrosophic sets, whose elements are characterized by one or more attributes, and each attribute can assume many values. Each attribute has a corresponding degree of appurtenance of the element to the set with respect to the given criteria. In order to obtain a better accuracy and for a more exact exclusion (partial order), a contradiction or dissimilarity degree is defined between each attribute value and the dominant attribute value. In this paper, entropy measures for plithogenic sets have been introduced. The requirements for any function to be an entropy measure of plithogenic sets are outlined in the axiomatic definition of the plithogenic entropy using the axiomatic requirements of neutrosophic entropy. Several new formulae for the entropy measure of plithogenic sets are also introduced. The newly introduced entropy measures are then applied to a multi-attribute decision making problem related to the selection of locations.

Lie Group Statistics and Lie Group Machine Learning Based on Souriau Lie Groups Thermodynamics & Koszul-Souriau-Fisher Metric: New Entropy Definition as Generalized Casimir Invariant Function in Coadjoint Representation.

In 1969, Jean-Marie Souriau introduced a “Lie Groups Thermodynamics” in Statistical Mechanics in the framework of Geometric Mechanics. This Souriau’s model considers the statistical mechanics of dynamic systems in their “space of evolution” associated to a homogeneous symplectic manifold by a Lagrange 2-form, and defines in case of non null cohomology (non equivariance of the coadjoint action on the moment map with appearance of an additional cocyle) a Gibbs density (of maximum entropy) that is covariant under the action of dynamic groups of physics (e.g., Galileo’s group in classical physics). Souriau Lie Group Thermodynamics was also addressed 30 years after Souriau by R.F. Streater in the framework of Quantum Physics by Information Geometry for some Lie algebras, but only in the case of null cohomology. Souriau method could then be applied on Lie groups to define a covariant maximum entropy density by Kirillov representation theory. We will illustrate this method for homogeneous Siegel domains and more especially for Poincaré unit disk by considering SU(1,1) group coadjoint orbit and by using its Souriau’s moment map. For this case, the coadjoint action on moment map is equivariant. For non-null cohomology, we give the case of Lie group SE(2). Finally, we will propose a new geometric definition of Entropy that could be built as a generalized Casimir invariant function in coadjoint representation, and Massieu characteristic function, dual of Entropy by Legendre transform, as a generalized Casimir invariant function in adjoint representation, where Souriau cocycle is a measure of the lack of equivariance of the moment mapping.

Network entropy using edge-based information functionals

Abstract In this article, we present a novel approach to analyse the structure of complex networks represented by a quantum graph. A quantum graph is a metric graph with a differential operator (including the edge-based Laplacian) acting on functions defined on the edges of the graph. Every edge of the graph has a length interval assigned to it. The structural information contents are measured using graph entropy which has been proved useful to analyse and compare the structure of complex networks. Our definition of graph entropy is based on local edge functionals. These edge functionals are obtained by a diffusion process defined using the edge-based Laplacian of the graph using the quantum graph representation. We first present the general framework to define graph entropy using heat diffusion process and discuss some of its properties for different types of network models. Second, we propose a novel signature to gauge the structural complexity of the network and apply the proposed method to different datasets.

Uniqueness and exponential mixing for the measure of maximal entropy for piecewise hyperbolic maps

<p style='text-indent:20px;'>For a class of piecewise hyperbolic maps in two dimensions, we propose a combinatorial definition of topological entropy by counting the maximal, open, connected components of the phase space on which iterates of the map are smooth. We prove that this quantity dominates the measure theoretic entropies of all invariant probability measures of the system, and then construct an invariant measure whose entropy equals the proposed topological entropy. We prove that our measure is the unique measure of maximal entropy, that it is ergodic, gives positive measure to every open set, and has exponential decay of correlations against Hölder continuous functions. As a consequence, we also prove a lower bound on the rate of growth of periodic orbits. The main tool used in the paper is the construction of anisotropic Banach spaces of distributions on which the relevant weighted transfer operator has a spectral gap. We then construct our measure of maximal entropy by taking a product of left and right maximal eigenvectors of this operator.

Towards Personalized Diagnosis of Glioblastoma in Fluid-Attenuated Inversion Recovery (FLAIR) by Topological Interpretable Machine Learning

Glioblastoma multiforme (GBM) is a fast-growing and highly invasive brain tumor, which tends to occur in adults between the ages of 45 and 70 and it accounts for 52 percent of all primary brain tumors. Usually, GBMs are detected by magnetic resonance images (MRI). Among MRI, a fluid-attenuated inversion recovery (FLAIR) sequence produces high quality digital tumor representation. Fast computer-aided detection and segmentation techniques are needed for overcoming subjective medical doctors (MDs) judgment. This study has three main novelties for demonstrating the role of topological features as new set of radiomics features which can be used as pillars of a personalized diagnostic systems of GBM analysis from FLAIR. For the first time topological data analysis is used for analyzing GBM from three complementary perspectives—tumor growth at cell level, temporal evolution of GBM in follow-up period and eventually GBM detection. The second novelty is represented by the definition of a new Shannon-like topological entropy, the so-called Generator Entropy. The third novelty is the combination of topological and textural features for training automatic interpretable machine learning. These novelties are demonstrated by three numerical experiments. Topological Data Analysis of a simplified 2D tumor growth mathematical model had allowed to understand the bio-chemical conditions that facilitate tumor growth—the higher the concentration of chemical nutrients the more virulent the process. Topological data analysis was used for evaluating GBM temporal progression on FLAIR recorded within 90 days following treatment completion and at progression. The experiment had confirmed that persistent entropy is a viable statistics for monitoring GBM evolution during the follow-up period. In the third experiment we developed a novel methodology based on topological and textural features and automatic interpretable machine learning for automatic GBM classification on FLAIR. The algorithm reached a classification accuracy up to 97%.

Static topological black hole with a nonminimal derivative coupling and a nonlinear electromagnetic field of Born-Infeld type

We consider scalar-tensor gravity with nonminimal derivative coupling and Born-Infeld electromagnetic field which is minimally coupled to gravity. Since cosmological constant is taken into account it allowed us not only derive static black hole with spherical horizon but also to obtain topological solutions with non-spherical horizons. The obtained metrics are thoroughly analyzed, namely for different distances and types of topology of horizon. To investigate singularities of the metrics Kretschmann scalar is used and it is shown that the character of singularity depends on the type of topology of horizon and dimension of space. We also investigate black hole's thermodynamics, namely we obtain and examine black hole's temperature. To derive the first law of black hole's thermodynamics Wald's approach is applied. Nonetheless this approach is well established, there is ambiguity in definition of black hole's entropy which can be resolved just by virtue of some independent approach.

The general formula of entropy and similarity measures for hesitant fuzzy linguistic term sets

In order to quantitatively characterize the uncertainty of hesitant fuzzy linguistic information, the entropy and similarity measures of hesitant fuzzy linguistic term sets are studied. First, the axiomatic definitions of entropy and similarity measures of hesitant fuzzy linguistic term sets are given, and general formulas of entropy and similarity measures for hesitant fuzzy linguistic term sets are proposed. Moreover, the algorithms for generating entropy and similarity of hesitant fuzzy linguistic term set are given. Then the relationship between entropy and similarity of hesitant fuzzy linguistic term set is studied, and the general formula of entropy based on similarity is proposed. These lay the theoretical foundation for flexible selection of entropy and similarity in multi-attribute decision making.

Studies of boundary entropy in AdS/BCFT

In this paper we review the AdS/BCFT proposal of Takayanagi for holographic description of systems with boundaries, in particular, boundary conformal field theories (BCFTs). Motivated by better understanding of the proposed duality we employ entanglement entropy as a probe of familiar properties of impurities and defects. Using the dual gravity description, we check that in two spacetime dimensions the impurity entropy does not depend on a particular state of the theory, which is a well-known CFT result. In three dimensions different, and not necessarily equivalent, definitions of the defect entropy can be given. We compute the entanglement entropy of a line defect at finite temperature and compare it with earlier calculations of the thermodynamical entropy. The results indicate that the entanglement entropy flows to the definition of the entropy as the Bekenstein–Hawking entropy associated to a portion of the black horizon, which we call impurity ‘shadow’. Geometric configurations, which we discuss, provide examples of RG flows of the defect entropies. We outline the connection between the geometric picture of the RG flows and examples of lattice calculations. We also discuss some new generalizations of the AdS/BCFT geometries.

New similarity and entropy measures of single-valued neutrosophic sets with applications in multi-attribute decision making

Information measures play a fundamental role in single-valued neutrosophic set (SVNS) theory. The main purpose of this paper is to study the similarity and entropy measures of SVNS with applications in multi-attribute decision making. We proposed the axiomatic definitions of similarity and entropy for single-valued neutrosophic values (SVNVs) with respect to a new kind of inclusion relation between SVNVs. On the basis of Hamming distance, cosine function and cotangent function, three similarity measures and three entropies for SVNVs are constructed. Then, we extended the definitions and construction methods of similarity and entropy for SVNVs to SVNSs by using some aggregation operators. Finally, by using the new similarity and entropy measures we presented a SVNSs based multi-attribute decision making method. It demonstrated that the new information measures presented in this study are applicable and efficient.

Entropy and Time.

The idea that entropy is associated with the “arrow of time” has its roots in Clausius’s statement on the Second Law: “Entropy of the Universe always increases.” However, the explicit association of the entropy with time’s arrow arises from Eddington. In this article, we start with a brief review of the idea that the “increase in entropy” is somehow associated with the direction in which time increases. Then, we examine three different, but equivalent definitions of entropy. We find that none of these definitions indicate any hint of a relationship between entropy and time. We can, therefore, conclude that entropy is a timeless quantity. We also discuss the reasons as to why some scientists went astray in associating entropy with time’s arrow. Finally, we shall discuss Boltzmann’s H-Theorem, which is viewed by many as a proof of the Second Law of Thermodynamics.

A Minimum Trust Discount Coefficient Model for Incomplete Information in Group Decision Making with Intuitionistic Fuzzy Soft Set

This article proposes a framework to deal with incomplete information in multiple criteria group decision making with intuitionistic fuzzy soft set. To do that, the weighted sum method for choice values and simple mathematical expectation method are extended to the case of obtained fuzzy soft set, and they are proved to have the same result in estimating the incomplete information. In order to reduce the system error in the estimating process cause by multiple decision information, a minimum trust discount coefficient model is established according to the relevant methods of evidence theory. Then, a new definition of entropy for intuitionistic fuzzy sets is introduced to determine the weights of group experts. Therefore, individual decision-making matrices are integrated into a comprehensive decision-making matrix by the integration operation formula of intuitionistic fuzzy soft matrix. The decision making is realized according to the difference of the score values of objects. Finally, the steps of this method are concluded, and one example is given to explain the application of this method.

Neoclassical Navier–Stokes Equations Considering the Gyftopoulos–Beretta Exposition of Thermodynamics

The seminal Navier–Stokes equations were stated even before the creation of the foundations of thermodynamics and its first and second laws. There is a widespread opinion in the literature on thermodynamic cycles that the Navier–Stokes equations cannot be taken as a thermodynamically correct model of a local “working fluid”, which would be able to describe the conversion of “heating” into “working” (Carnot’s type cycles) and vice versa (Afanasjeva’s type cycles). Also, it is overall doubtful that “cycle work is converted into cycle heat” or vice versa. The underlying reason for this situation is that the Navier–Stokes equations come from a time when thermodynamic concepts such as “internal energy” were still poorly understood. Therefore, this paper presents a new exposition of thermodynamically consistent Navier–Stokes equations. Following that line of reasoning—and following Gyftopoulos and Beretta’s exposition of thermodynamics—we introduce the basic concepts of thermodynamics such as “heating” and “working” fluxes. We also develop the Gyftopoulos and Beretta approach from 0D into 3D continuum thermodynamics. The central role within our approach is played by “internal energy” and “energy conversion by fluxes.” Therefore, the main problem of exposition relates to the internal energy treated here as a form of “energy storage.” Within that context, different forms of energy are discussed. In the end, the balance of energy is presented as a sum of internal, kinetic, potential, chemical, electrical, magnetic, and radiation energies in the system. These are compensated by total energy flux composed of working, heating, chemical, electrical, magnetic, and radiation fluxes at the system boundaries. Therefore, the law of energy conservation can be considered to be the most important and superior to any other law of nature. This article develops and presents in detail the neoclassical set of Navier–Stokes equations forming a thermodynamically consistent model. This is followed by a comparison with the definition of entropy (for equilibrium and non-equilibrium states) within the context of available energy as proposed in the Gyftopoulos and Beretta monograph. The article also discusses new possibilities emerging from this “continual” Gyftopoulos–Beretta exposition with special emphasis on those relating to extended irreversible thermodynamics or Van’s “universal second law”.

On the gravitational entropy of accelerating black holes

In this paper, we have examined the validity of a proposed definition of gravitational entropy in the context of accelerating black hole solutions of the Einstein field equations, which represent the realistic black hole solutions. We have adopted a phenomenological approach proposed in Rudjord et al. [Phys. Scr. 77, 055901 (2008)] and expanded by Romero et al. [Int. J. Theor. Phys. 51, 925 (2012)], in which the Weyl curvature hypothesis is tested against the expressions for the gravitational entropy. Considering the [Formula: see text]-metric for the accelerating black holes, we have evaluated the gravitational entropy and the corresponding entropy density for four different types of black holes, namely, nonrotating black hole, nonrotating charged black hole, rotating black hole and rotating charged black hole. We end up by discussing the merits of such an analysis and the possible reason of failure in the particular case of rotating charged black hole and comment on the possible resolution of the problem.

The conditional average entropies

This paper introduces two more definitions of the conditional average entropy. Some properties of the three definitions are studied and some mistakes in the preceding literature are corrected.

Quantum corrections to the entropy and its application in the study of quantum Carnot engines.

Entropy is one of the most basic concepts in thermodynamics and statistical mechanics. The most widely used definition of statistical mechanical entropy for a quantum system is introduced by von Neumann. While in classical systems, the statistical mechanical entropy is defined by Gibbs. The relation between these two definitions of entropy is still not fully explored. In this work, we study this problem by employing the phase-space formulation of quantum mechanics. For those quantum states having well-defined classical counterparts, we study the quantum-classical correspondence and quantum corrections of the entropy. We expand the von Neumann entropy in powers of ℏ by using the phase-space formulation, and the zeroth-order term reproduces the Gibbs entropy. We also obtain the explicit expression of the quantum corrections of the entropy. Moreover, we find that for the thermodynamic equilibrium state, all terms odd in ℏ are exactly zero. As an application, we derive quantum corrections for the net work extraction during a quantum Carnot cycle. Our results bring important insights into the understanding of quantum entropy and may have potential applications in the study of quantum heat engines.

GLoG: Laplacian of Gaussian for Spatial Pattern Detection in Spatio-Temporal Data.

Boundary detection has long been a fundamental tool for image processing and computer vision, supporting the analysis of static and time-varying data. In this work, we built upon the theory of Graph Signal Processing to propose a novel boundary detection filter in the context of graphs, having as main application scenario the visual analysis of spatio-temporal data. More specifically, we propose the equivalent for graphs of the so-called Laplacian of Gaussian edge detection filter, which is widely used in image processing. The proposed filter is able to reveal interesting spatial patterns while still enabling the definition of entropy of time slices. The entropy reveals the degree of randomness of a time slice, helping users to identify expected and unexpected phenomena over time. The effectiveness of our approach appears in applications involving synthetic and real data sets, which show that the proposed methodology is able to uncover interesting spatial and temporal phenomena. The provided examples and case studies make clear the usefulness of our approach as a mechanism to support visual analytic tasks involving spatio-temporal data.

Thermodynamics of Quantum Causal Models: An Inclusive, Hamiltonian Approach

Operational quantum stochastic thermodynamics is a recently proposed theory to study the thermodynamics of open systems based on the rigorous notion of a quantum stochastic process or quantum causal model. In there, a stochastic trajectory is defined solely in terms of experimentally accessible measurement results, which serve as the basis to define the corresponding thermodynamic quantities. In contrast to this observer-dependent point of view, a `black box', which evolves unitarily and can simulate a quantum causal model, is constructed here. The quantum thermodynamics of this big isolated system can then be studied using widely accepted arguments from statistical mechanics. It is shown that the resulting definitions of internal energy, heat, work, and entropy have a natural extension to the trajectory level. The canonical choice of them coincides with the proclaimed definitions of operational quantum stochastic thermodynamics, thereby providing strong support in favour of that novel framework. However, a few remaining ambiguities in the definition of stochastic work and heat are also discovered and in light of these findings some other proposals are reconsidered. Finally, it is demonstrated that the first and second law hold for an even wider range of scenarios than previously thought, covering a large class of quantum causal models based solely on a single assumption about the initial system-bath state.

Limsup is needed in the definitions of topological entropy via spanning or separation numbers

ABSTRACT The notion of topological entropy can be conceptualized in terms of the number of forward trajectories that are distinguishable at resolution ϵ within T time units. It can then be formally defined as a limit of a limit superior that involves either covering numbers, or separation numbers, or spanning numbers. If covering numbers are used, the limit superior reduces to a limit. While it has been generally believed that the latter may not necessarily be the case when the definition is based on separation or spanning numbers, no actual counterexamples appear to have been previously known. Here we fill this gap in the literature by constructing such counterexamples.

Information density, structure and entropy in equilibrium and non-equilibrium systems

During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the statistical mechanical version of the second law of thermodynamics, and it provides an interpretation of entropy in terms of probabilities. However, we cannot rely on the statistical-mechanical expressions for entropy in systems that are far from equilibrium. In this paper, we compare various extensions of the definition of entropy, which have been proposed for non-equilibrium systems. It has recently been proposed that measures of information density may serve to quantify entropy in both equilibrium and non-equilibrium systems. We propose a new ‘bit-wise’ method to measure the information density for off-lattice systems. This method does not rely on coarse-graining of the particle coordinates. We then compare different estimates of the system entropy, based on information density and on the structural properties of the system, and check if the various entropies are mutually consistent and, importantly, whether they can detect non-trivial ordering phenomena. We find that, except for simple (one-dimensional) cases, the different methods yield answers that are at best qualitatively similar, and often not even that, although in several cases, the different entropy estimates do detect ordering phenomena qualitatively. Our entropy estimates based on bit-wise data compression contain no adjustable scaling factor, and show large quantitative differences with the thermodynamic entropy obtained from equilibrium simulations. Hence, our results suggest that, at present, there is not yet a single, structure-based entropy definition that has general validity for equilibrium and non-equilibrium systems.