Entropy Of Distribution Research Articles

A Source Seeking Method for the Implicit Information Field Based on a Balanced Searching Strategy

To address the issue of low efficiency in source seeking within implicit information fields, this paper proposes an autonomous sourcing method based on a balanced search strategy inspired by biological homing behaviors. At the outset of the research, the task of source seeking boiled down to a multi-objective convergence problem. By utilizing feasibility search behaviors as individual samples in evolutionary population, drawing on the principles of evolutionary algorithms, motion searching was integrated with population evolution to guide carriers towards completing source seeking tasks by solving multi-objective problems. Furthermore, the distribution entropy was also considered to measure the searching bias in the process of source seeking. In combination with the requirements of the source seeking process, a new method for balanced searching was designed. Ultimately, through theoretical analysis and simulation verification, we confirmed the effectiveness and rationality of this proposed method.

Gene expression programming with dual strategies and neighborhood search for symbolic regression problems

As an evolutionary algorithm for automatic generation of computer programs, Gene expression programming (GEP) has received a lot of attention because it can generate concise and highly accurate solutions. However, GEP also has some unsatisfactory aspects, such as too many control parameters and a lack of balance between exploration and exploitation. Therefore, a new GEP with dual strategies and neighborhood search (DNSGEP) is proposed in this paper for symbolic regression problems. As the core part, a new multi-parent crossover operator with neighborhood search is proposed to replace the original transposition and recombination operators in GEP, which greatly reduces the excessive control parameters in GEP. To assist the multi-parent crossover operator in selecting parents, selection weight is used to assess the quality of individuals in the population. The multi-parent crossover operator combines selection weight to learn more from the better parents. In order to satisfy the different needs of global search and local search of the algorithm, two different crossover strategies are extended on the basis of the multi-parent crossover operator. Furthermore, a population distribution entropy strategy to GEP is proposed for determining when the algorithm should choose which crossover strategy to use. DNSGEP is compared with GEP and three GEP variants on 14 symbolic regression problems and 6 even parity problems. Experimental results show that DNSGEP has the success rate and accuracy that are fully superior to GEP, GEPADF and OGEP, not inferior to SLGEP.

Maximum entropy distributions of dark matter in ΛCDM cosmology

Context. Small-scale challenges to ΛCDM cosmology require a deeper understanding of dark matter physics. Aims. This paper aims to develop the maximum entropy distributions for dark matter particle velocity (denoted by X), speed (denoted by Z), and energy (denoted by E) that are especially relevant on small scales where system approaches full virialization. Methods. For systems involving long-range interactions, a spectrum of halos of different sizes is required to form to maximize system entropy. While the velocity in halos can be Gaussian, the velocity distribution throughout the entire system, involving all halos of different sizes, is non-Gaussian. With the virial theorem for mechanical equilibrium, we applied the maximum entropy principle to the statistical equilibrium of entire system, such that the maximum entropy distribution of velocity (the X distribution) could be analytically derived. The halo mass function was not required in this formulation, but it did indeed result from the maximum entropy. Results. The predicted X distribution involves a shape parameter α and a velocity scale, v0. The shape parameter α reflects the nature of force (α → 0 for long-range force or α → ∞ for short-range force). Therefore, the distribution approaches Laplacian with α → 0 and Gaussian with α → ∞. For an intermediate value of α, the distribution naturally exhibits a Gaussian core for v ≪ v0 and exponential wings for v ≫ v0, as confirmed by N-body simulations. From this distribution, the mean particle energy of all dark matter particles with a given speed, v, follows a parabolic scaling for low speeds (∝v2 for v ≪ v0 in halo core region, i.e., “Newtonian”) and a linear scaling for high speeds (∝v for v ≫ v0 in halo outskirt, i.e., exhibiting “non-Newtonian” behavior due to long-range gravity). We compared our results against N-body simulations and found a good agreement.

Predicting the geographical distribution of Impatiens parasitica (Balsaminaceae), an epiphytic species endemic to the Southern Western Ghats, India

Impatiens parasitica Bedd. is an epiphytic balsam, endemic to the southern Western Ghats region of India. Maximum entropy distribution modelling (MaxEnt) was used for predicting the potential habitat suitability for this species. The Area Under the receiver operating characteristics Curve (AUC) value for training and test (0.95 and 0.817) is higher than 0.8, hence the model can be accepted. Jackknife test illustrates that temperature seasonality is the most contributed variable and Terrain Ruggedness Index (TRI) is the least contributed variable. The model predicted high suitability in the evergreen and semi-evergreen forests of the southern Western Ghats.

Rényi Entropy for Past Lifetime Distributions with Application in Inactive Coherent Systems

In parallel with the concept of Rényi entropy for residual lifetime distributions, the Rényi entropy of inactivity time of lifetime distributions belonging to asymmetric distributions is a useful measure of independent interest. For a system that turns out to be inactive in time t, the past entropy is considered as an uncertainty measure for the past lifetime distribution. In this study, we consider a coherent system that includes n components and has the property that all the components of the system have failed at time t. To assess the predictability of the coherent system’s lifetime, we use the system’s signature to determine the Rényi entropy of its past lifetime. We study several analytical results, including expressions, bounds, and order properties for this measure.

Combating Mode Collapse via Offline Manifold Entropy Estimation

Generative Adversarial Networks (GANs) have shown compelling results in various tasks and applications in recent years. However, mode collapse remains a critical problem in GANs. In this paper, we propose a novel training pipeline to address the mode collapse issue of GANs. Different from existing methods, we propose to generalize the discriminator as feature embedding and maximize the entropy of distributions in the embedding space learned by the discriminator. Specifically, two regularization terms, i.e., Deep Local Linear Embedding (DLLE) and Deep Isometric feature Mapping (DIsoMap), are introduced to encourage the discriminator to learn the structural information embedded in the data, such that the embedding space learned by the discriminator can be well-formed. Based on the well-learned embedding space supported by the discriminator, a non-parametric entropy estimator is designed to efficiently maximize the entropy of embedding vectors, playing as an approximation of maximizing the entropy of the generated distribution. By improving the discriminator and maximizing the distance of the most similar samples in the embedding space, our pipeline effectively reduces the mode collapse without sacrificing the quality of generated samples. Extensive experimental results show the effectiveness of our method which outperforms the GAN baseline, MaF-GAN on CelebA (9.13 vs. 12.43 in FID) and surpasses the recent state-of-the-art energy-based model on the ANIMEFACE dataset (2.80 vs. 2.26 in Inception score).

Evolution of many-body systems under ancilla quantum measurements

Measurement-induced phase transitions are the subject of intense current research, both from an experimental and a theoretical perspective. We explore the concept of implementing quantum measurements by coupling a many-body lattice system to an ancillary degree of freedom (implemented using two additional sites), on which projective measurements are performed. We analyze the effect of repeated (``stroboscopic'') measurements on the dynamical correlations of interacting hard-core bosons in a one-dimensional chain. An important distinctive ingredient of the protocol is the fact that the detector ancillae are not reinitialized after each measurement step. The detector thus maintains memory of the accumulated influence by the measured correlated system. Initially, we consider a model in which the ancilla is coupled to a single lattice site. This setup allows obtaining information about the system through Rabi oscillations in the ancillary degrees of freedom, modulated by the ancilla-system interaction. The statistics of quantum trajectories exhibits a ``quantum-Zeno-valve effect'' that occurs when the measurement becomes strong, with sharp branching between low and high entanglement. We proceed by extending numerical simulations to the case of two ancillae and, then, to measurements on all sites. With this realistic measurement apparatus, we find evidence of a disentangling-entangling measurement-induced transition as was previously observed in more abstract models. The dynamics features a broad distribution of the entanglement entropy.

A multi-scale convolutional neural network-based model for clustering economic risk detection.

After the public health event of COVID-19, more academics are looking into how to predict combined economic hazards associated with public health incidents. There are currently just a few approaches for detecting aberrant behaviour in aggregated financial risk, and most only work after the economic risk has already been inappropriately aggregated. As a result, we provide a multi-scale convolutional neural network-based model for clustering financial risk anomaly detection (MCNN). First, we use MCNN to train a model for counting economic risks that are used to evaluate aberrant risk aggregating data. Second, we can use the test results to extract the financial risk statistics and economic risk precursor coordinate points. Then, we calculate the economic risk distribution entropy, distance, potential energy and density. To train the three elements of the development state and create the prediction model, we finally use the particle swarm optimization-based extreme learning machine (PSO-ELM). The results of the experiments demonstrate that, in comparison to existing algorithms, our model can efficiently realize early warning and detect abnormal behaviours of aggregated economic risks with high timeliness. Additionally, our method achieves a forecast accuracy of 97.68% and can give additional time to take emergency action.

EMD-Based Energy Spectrum Entropy Distribution Signal Detection Methods for Marine Mammal Vocalizations

To develop a passive acoustic monitoring system for diversity detection and thereby adapt to the challenges of a complex marine environment, this study harnesses the advantages of empirical mode decomposition in analyzing nonstationary signals and introduces energy characteristics analysis and entropy of information theory to detect marine mammal vocalizations. The proposed detection algorithm has five main steps: sampling, energy characteristics analysis, marginal frequency distribution, feature extraction, and detection, which involve four signal feature extraction and analysis algorithms: energy ratio distribution (ERD), energy spectrum distribution (ESD), energy spectrum entropy distribution (ESED), and concentrated energy spectrum entropy distribution (CESED). In an experiment on 500 sampled signals (blue whale vocalizations), in the competent intrinsic mode function (IMF2) signal feature extraction function distribution of ERD, ESD, ESED, and CESED, the areas under the curves (AUCs) of the receiver operating characteristic (ROC) curves were 0.4621, 0.6162, 0.3894, and 0.8979, respectively; the Accuracy scores were 49.90%, 60.40%, 47.50%, and 80.84%, respectively; the Precision scores were 31.19%, 44.89%, 29.44%, and 68.20%, respectively; the Recall scores were 42.83%, 57.71%, 36.00%, and 84.57%, respectively; and the F1 scores were 37.41%, 50.50%, 32.39%, and 75.51%, respectively, based on the threshold of the optimal estimated results. It is clear that the CESED detector outperforms the other three detectors in signal detection and achieves efficient sound detection of marine mammals.

Entropy defect in thermodynamics

This paper describes the physical foundations of the newly discovered “entropy defect” as a basic concept of thermodynamics. The entropy defect quantifies the change in entropy caused by the order induced in a system through the additional correlations among its constituents when two or more subsystems are assembled. This defect is closely analogous to the mass defect that arises when nuclear particle systems are assembled. The entropy defect determines how the entropy of the system compares to its constituent’s entropies and stands on three fundamental properties: each constituent’s entropy must be (i) separable, (ii) symmetric, and (iii) bounded. We show that these properties provide a solid foundation for the entropy defect and for generalizing thermodynamics to describe systems residing out of the classical thermal equilibrium, both in stationary and nonstationary states. In stationary states, the consequent thermodynamics generalizes the classical framework, which was based on the Boltzmann–Gibbs entropy and Maxwell–Boltzmann canonical distribution of particle velocities, into the respective entropy and canonical distribution associated with kappa distributions. In nonstationary states, the entropy defect similarly acts as a negative feedback, or reduction of the increase of entropy, preventing its unbounded growth toward infinity.

Bayesian Estimations of Shannon Entropy and Rényi Entropy of Inverse Weibull Distribution

In this paper, under the symmetric entropy and the scale squared error loss functions, we consider the maximum likelihood (ML) estimation and Bayesian estimation of the Shannon entropy and Rényi entropy of the two-parameter inverse Weibull distribution. In the ML estimation, the dichotomy is used to solve the likelihood equation. In addition, the approximation confidence interval is given by the Delta method. Because the form of estimation results is more complex in the Bayesian estimation, the Lindley approximation method is used to achieve the numerical calculation. Finally, Monte Carlo simulations and a real dataset are used to illustrate the results derived. By comparing the mean square error between the estimated value and the real value, it can be found that the performance of ML estimation of Shannon entropy is better than that of Bayesian estimation, and there is no significant difference between the performance of ML estimation of Rényi entropy and that of Bayesian estimation.

A highly generalized classifier for osteoporosis radiography based on multiscale fractal, lacunarity, and entropy distributions.

Background: Osteoporosis is a common degenerative disease with high incidence among aging populations. However, in regular radiographic diagnostics, asymptomatic osteoporosis is often overlooked and does not include tests for bone mineral density or bone trabecular condition. Therefore, we proposed a highly generalized classifier for osteoporosis radiography based on the multiscale fractal, lacunarity, and entropy distributions. Methods: We collected a total of 104 radiographs (92 for training and 12 for testing) of lumbar spine L4 and divided them into three groups (normal, osteopenia, and osteoporosis). In parallel, 174 radiographs (116 for training and 58 for testing) of calcaneus from health and osteoporotic fracture groups were collected. The texture feature data of all the radiographs were pulled out and analyzed. The Davies-Bouldin index was applied to optimize hyperparameters of feature counting. Neighborhood component analysis was performed to reduce feature dimension and increase generalization. A support vector machine classifier was trained with only the most effective six features for each binary classification scenario. The accuracy and sensitivity performance were estimated by calculating the area under the curve. Results: Interpretable feature trends of osteoporotic pathological changes were depicted. On the spine test dataset, the accuracy and sensitivity of binary classifiers were 0.851 (95% CI: 0.730-0.922), 0.813 (95% CI: 0.718-0.878), and 0.936 (95% CI: 0.826-1) for osteoporosis diagnosis; 0.721 (95% CI: 0.578-0.824), 0.675 (95% CI: 0.563-0.772), and 0.774 (95% CI: 0.635-0.878) for osteopenia diagnosis; and 0.935 (95% CI: 0.830-0.968), 0.928 (95% CI: 0.863-0.963), and 0.910 (95% CI: 0.746-1) for osteoporosis diagnosis from osteopenia. On the calcaneus test dataset, they were 0.767 (95% CI: 0.629-0.879), 0.672 (95% CI: 0.545-0.793), and 0.790 (95% CI: 0.621-0.923) for osteoporosis diagnosis. Conclusion: This method showed the capacity of resisting disturbance on lateral spine radiographs and high generalization on the calcaneus dataset. Pixel-wise texture features not only helped to understand osteoporosis on radiographs better but also shed new light on computer-aided osteopenia and osteoporosis diagnosis.

A (Slightly) Improved Approximation Algorithm for Metric TSP

In “An Improved Approximation Algorithm for TSP,” Karlin, Klein, and Oveis Gharan design the first improvement over the classical 1.5 approximation algorithm of Christofides-Serdyukov after more than 40 years. Their algorithm first chooses a random spanning tree from the maximum entropy distribution of spanning trees with marginals equal to the optimum LP solution of TSP, and then, similar to Christofides’ algorithm, it adds the minimum cost matching on the odd degree vertices of the tree. To analyze their simple algorithms, they prove and exploit new tools from the theory of strongly Rayleigh distributions.

The growth and size of orogenic gold systems: probability and dynamical behaviour

Every nonlinear system grows by increments, and the final probability distributions for components of that system emerge from an amalgamation of these increments. The resulting probability distribution depends on the constraints imposed on each increment by the physical and chemical processes that produce the system. Hence there is the potential that the observed probability distribution can reveal information on these processes. Complex systems that grow by competition between the supply and consumption of energy and mass have growth laws that are cumulative probability distributions for their component parts that reflect such competition. We show that the type of probability distribution is characteristic of the endowment of orogenic gold deposits with the sequence: Weibull → Fréchet → gamma → log normal representative of increasing endowment. Further, the differential entropy of the probability distribution is indicative of the quality of the deposit, with low-quality deposits represented by high entropy and high-quality deposits represented by low or negative entropy. The type of probability distribution gives an indication of the processes that operated to produce the deposit. These conclusions hold for mineralisation as well as for the associated alteration assemblages. We suggest that the probability distribution for the mineralisation or the alteration assemblage gives a good indication of the endowment and quality of a deposit from a single drill hole. KEY POINTS A single drill hole from a deposit can provide information on endowment and organisation. Weibull → Fréchet → gamma → log normal probability distributions are representative of increasing gold endowment. The differential entropies of these distributions characterise the organisation of the system.

Extended hydrodynamical models for plasmas

We propose an extended hydrodynamical model for plasmas, based on the moments of the electron distribution function which satisfies the Fokker–Planck–Landau (FPL) transport equation. The equations for the moments can be obtained by multiplying the FPL equation by the corresponding weight functions and integrating over the velocity space. The moments are decomposed in their convective and non–convective parts and closure relations for the fluxes and production terms can be obtained by using the maximum entropy distribution function, which depends on Lagrangian multipliers. These latter can be expressed in terms of the state variables by imposing the constraints that the maximum entropy distribution function reproduces the moments chosen as state variables. In particular, we will concentrate on the 13-moment system. As a first application, we treat the case of the relaxation towards equilibrium of a homogeneous plasma with a temperature anisotropy, showing that the results are in good agreement with those obtained by means of the Kogan solution of the kinetic equation.

A Study of Holographic Dark Energy Models with Configuration Entropy

The holographic dark energy models provide an alternative description of dark energy. These models are motivated by the possible application of the holographic principle to the dark energy problem. In this work, we present a theoretical study of the one parameter Li holographic dark energy and the two parameter Barrow holographic dark energy models using configuration entropy of the matter distribution in the universe. The configuration entropy rate exhibits a distinct minimum at a specific scale factor that corresponds to the epoch, beyond which dark energy takes a driving role in the accelerated expansion of the universe. We find that the location of the minimum and magnitude of the entropy rate at the minimum are sensitive to the parameters of the models. We find the best fit relations between these quantities and the parameters of each model. We propose that these relations can be used to constrain the parameters of the holographic dark energy models from future observations such as the SKA. Our study suggests that the signature of a large quantum gravitational effect on the future event horizon can be detected from measurements of the configuration entropy of the matter distribution at multiple redshifts.

Information entropy and fragmentation functions

Several groups have recently investigated the flow of information in high-energy collisions, from the entanglement entropy of the proton yielding classical Shannon entropy of its parton distribution functions (pdfs), through jet splitting generating entropy, to the entropy distribution in hadron decays.Lacking in the literature is a discussion of the information entropy of fragmentation functions (FFs) in the instances where they can be considered as probability distributions, and we here provide it. We find that this entropy is a single, convenient number to characterize future progress in the extraction of fragmentation functions.We also deploy the related Kullback-Leibler divergence between two distributions to assess existing relations among FFs and parton distribution functions (pdfs) such as that of Barone, Drago and Ma. From a couple of current parametrizations of FFs, we do not find supporting empirical evidence for the relation, although it is possible that FFs and pdfs have similar power-laws near the x=1 endpoint.

A Novel Method to the Aging Feature Extraction of Viscoelastic Sandwich Structures

The viscoelastic sandwich structure exhibits highly nonlinear dynamic characteristics, and minimal changes in signal caused by aging state alterations, thus posing a challenge for extracting its aging characteristics. A method is proposed for extracting aging features from viscoelastic sandwich structures using variational mode decomposition (VMD) and permutation entropy (PE). First, the original dynamic response signal is decomposed by VMD to acquire several intrinsic mode functions (IMF). Then the permutation entropy characteristics of each IMF are calculated. The research results show that there are obvious differences in the distribution of permutation entropy corresponding to each IMF component, which indicates that the permutation entropy feature extracted based on the variational modal decomposition is more susceptive to the change of the aging state of the structure, thus verifying the availability of this method for feature extraction.

Opportune warning of COVID-19 in a Mexican health care worker cohort: Discrete beta distribution entropy of smartwatch physiological records

We present a statistical study of heart rate, step cadence, and sleep stage registers of health care workers in the Hospital General de México “Dr. Eduardo Liceaga” (HGM), monitored continuously and non-invasively during the COVID-19 contingency from May to October 2020, using the Fitbit Charge 3® Smartwatch device. The HGM-COVID cohort consisted of 115 participants assigned to areas of COVID-19 exposure. We introduce a novel biomarker for an opportune signal for the likelihood of SARS-CoV-2 infection based on the Shannon Entropy of the Discrete Generalized Beta Distribution fit of rank ordered smartwatch registers. Our statistical test indicated infection for 94% of patients confirmed by positive polymer chain reaction (PCR+) test, 47% before the test, and 47% in coincidence. These results required innovative data preprocessing for the definition of a new biomarker index. The statistical method parameters are data-driven, confidence estimates were calibrated based on sensitivity tests using appropriately derived surrogate data as a benchmark. Our surrogate tests can also provide a benchmark for comparing results from other anomaly detection methods (ADMs). Biomarker comparison of the negative Immunoglobulin G Antibody (IgG-) subgroup with the PCR+ subgroup showed a statistically significant difference (p < 0.01, effect size = 1.44). The distribution of the uninfected population had a lower median and less dispersion than the PCR+ population. A retrospective study of our results confirmed that the biomarker index provides an early warning of the likelihood of COVID-19, even several days before the onset of symptoms or the PCR+ test request. The method can be calibrated for the analysis of different SARS-CoV-2 strains, the effect of vaccination, and previous infections. Furthermore, our biomarker screening could be implemented to provide general health profiles for other population sectors based on physiological signals from smartwatch wearable devices.

A working-condition-robust method for two-stage health measurement of rolling bearings based on energy entropy distribution and dynamic adversarial transfer network

Accurate and robust health measurement for rolling bearings under variable working conditions has great significance in guaranteeing the safe and stable operation of rotating machinery. In this paper, a two-stage and working-condition-robust health measurement method is proposed, systematically blending energy entropy theory, a deep-learning approach and transfer-learning technology. In the first stage, a state boundary of energy entropy is systematically deduced based on an adaptive variational mode decomposition (VMD) improved fruit fly optimization algorithm (IFOA) and the principle of statistical analysis to detect abnormal states in bearings, where the IFOA is developed to search for the optimal parameters of the VMD with high efficiency. In the second stage, if a fault exists, a hybrid robust auto-encoder adopting a multi-layer and deep structure is constructed to strengthen the feature extraction capacity and automatically capture valuable and robust fault features from original samples. Considering the insufficiently labeled samples and significant data distribution discrepancy, a novel dynamic adversarial transfer network (DATN) is designed to extract the transferable and domain-invariant features between source and target datasets and achieve accurate fault identification. Specifically, a dynamic adversarial coefficient based on Wasserstein distance is provided in the DATN to quantitatively evaluate the relative importance of marginal and conditional distributions. Extensive experiments on two rolling bearing datasets validate the superior performance of the proposed method compared with other state-of-the-art identification models and transfer-learning approaches.