Relative Entropy Research Articles

The General Theory of Scientific Variability for Technological Evolution

The proposed general theory of scientific variability for technological evolution explains one of the drivers of technological change for economic progress in human society. Variability is the predisposition of the elements in systems to assume different values over time and space. In biology, the variability is basic to explaining differences and development in organisms. In economics of technical change, the effects of variability within research fields on evolutionary dynamics of related technologies are unknown. In a broad analogy with the principles of biology, suggested theoretical framework here can clarify a basic driver of technological evolution: the variability within research fields can explain the dynamics of scientific development and technological evolution. The study sees whether statistical evidence supports the hypothesis that the rate of growth of scientific and technological fields can be explained by the level of variability within scientific fields. The validation is based on emerging research fields in quantum technologies: quantum imaging, quantum meteorology, quantum sensing, and quantum optics. Statistical evidence seems in general to support the hypothesis stated that the rate of growth can be explained by the level of scientific variability within research fields, measured with the relative entropy (indicating the dispersion of scientific topics in a research field underlying a specific technology). Nonparametric correlation with Spearman’s rho shows a positive coefficient of 0.80 between entropy measures and rates of growth between scientific and technological fields. The linear model of the relation between rate of growth and scientific variability reveals a coefficient of regression equal to 1.63 (R2 = 0.60). The findings here suggest a general law that variability within research fields positively drives scientific development and technological evolution. In particular, a higher variability within research fields can support a high rate of growth in scientific development and technological evolution. The proposed general theory of scientific variability is especially relevant in turbulent environments of technology-based competition to clarify a basic determinant of technological development to design strategies of technological forecasting and management of promising innovations.

The informativeness of [C ii] line-intensity mapping as a probe of the H i content and metallicity of galaxies at the end of reionization

ABSTRACT Line-intensity mapping (LIM) experiments coming online now will survey fluctuations in aggregate emission in the [C ii] ionized carbon line from galaxies at the end of reionization. Experimental progress must be matched by theoretical reassessments of approaches to modelling and the information content of the signal. We present a new model for the halo–[C ii] connection, building upon results from the Feedback In Realistic Environments simulations suggesting that gas mass and metallicity most directly determine [C ii] luminosity. Applying our new model to an ensemble of peak-patch halo light-cones, we generate new predictions for the [C ii] LIM signal at z ≳ 6. We expect a baseline 4000-h LIM survey from the Cerro Chajnantor Atacama Telescope facility to have the fundamental sensitivity to detect the [C ii] power spectrum at a significance of 5σ at z ∼ 6, with an extended or successor Stage 2 experiment improving significance to 48σ at z ∼ 6 and achieving 11σ at z ∼ 7.5. Cross-correlation through stacking, simulated against a mock narrow-band Lyman-break galaxy survey, would yield a strong detection of the radial profile of cosmological [C ii] emission surrounding star-forming galaxies. We also analyse the role of a few of our model’s parameters through the pointwise relative entropy (PRE) of the distribution of [C ii] intensities. While the PRE signature of different model parameters can become degenerate or diminished after factoring in observational distortions, various parameters do imprint themselves differently on the one-point statistics of the intrinsic signal. Further work can pave the way to access this information and distinguish different sources of non-Gaussianity in the [C ii] LIM observation.

Coherence generation with Hamiltonians

We explore methods to generate quantum coherence through unitary evolutions, by introducing and studying the coherence generating capacity of Hamiltonians. This quantity is defined as the maximum derivative of coherence that can be achieved by a Hamiltonian. By adopting the relative entropy of coherence as our figure of merit, we evaluate the maximal coherence generating capacity with the constraint of a bounded Hilbert-Schmidt norm for the Hamiltonian. Our investigation yields closed-form expressions for both Hamiltonians and quantum states that induce the maximal derivative of coherence under these conditions. Specifically, for qubit systems, we solve this problem comprehensively for any given Hamiltonian, identifying the quantum states that lead to the largest coherence derivative induced by the Hamiltonian. Our investigation enables a precise identification of conditions under which quantum coherence is optimally enhanced, offering valuable insights for the manipulation and control of quantum coherence in quantum systems.

Dynamics of quantum coherence and non-classical correlations between non-interacting two 2-level atoms in thermal baths

This paper delves into the exploration of quantum resources within a two-qubit system composed of two 2-level atoms connected to distinct thermal baths. Various quantifiers are employed to evaluate different aspects of the system’s quantum characteristics. Specifically, the [Formula: see text]-norm and relative entropy of coherence ([Formula: see text] and [Formula: see text]) are utilized to gauge quantum coherence, while local quantum Fisher information (LQFI) is used to quantify non-classical correlations within the system. The findings suggest that the amount of quantum correlations and coherence decline as the spontaneous emission rate [Formula: see text] and the mean thermal photon number n increase. However, it is observed that manipulating parameters defining the initial state of the two-level atoms system can enhance non-classical correlations and quantum coherence between the two atoms. Additionally, it is noted that these three key metrics entirely vanish in the asymptotic limit of time. Our research underscores the importance of precisely adjusting the parameters of the initial system state, which is prepared in an extended Werner-like state (EWL), to protect quantum resources shared between the two atoms from environmental influences.

Continuity of the relative entropy of resource

We present a criterion to establish the local continuity of the relative entropy of resource, i.e. the relative entropy distance to the set of free states, in any quantum resource theory. Several basic corollaries of this criterion are presented. Applications to the relative entropy of entanglement in multipartite quantum systems are considered. It is shown, in particular, that local continuity of any relative entropy of multipartite entanglement follows from local continuity of the quantum mutual information.

Viewpoint Selection for 3D-Games with f-Divergences.

In this paper, we present a novel approach for the optimal camera selection in video games. The new approach explores the use of information theoretic metrics f-divergences, to measure the correlation between the objects as viewed in camera frustum and the ideal or target view. The f-divergences considered are the Kullback-Leibler divergence or relative entropy, the total variation and the χ2 divergence. Shannon entropy is also used for comparison purposes. The visibility is measured using the differential form factors from the camera to objects and is computed by casting rays with importance sampling Monte Carlo. Our method allows a very fast dynamic selection of the best viewpoints, which can take into account changes in the scene, in the ideal or target view, and in the objectives of the game. Our prototype is implemented in Unity engine, and our results show an efficient selection of the camera and an improved visual quality. The most discriminating results are obtained with the use of Kullback-Leibler divergence.

PDHL-MARCOS method based on cumulative prospect theory for MAGDM and its application in network security evaluation

The increasingly large and complex network brings convenience to our life and production, but it also brings potential dangers. Any minor vulnerability may lead to failure of application software configuration, software maintenance, user management and system management. Eventually, our network is interrupted, and even life and production data are leaked. Therefore, establishing a scientific and effective network security evaluation model is an effective way to supervise, manage and improve network security; It is also an effective measure to reduce the loss of life and production. Therefore, we first selected six factors closely related to network security to form a complete network security evaluation index system. Then, the utility function in the traditional measurement of alternatives and ranking according to compromise solution (MARCOS) method is improved by using the value function in cumulative prospect theory (CPT), and a PDHL-MARCOS evaluation model that can reflect the decision-makers’ (DMs’) attitudes towards gains and losses is obtained in the probabilistic double hierarchy linguistic term set (PDHLTS). In addition, the relative entropy (RE) method is also used to obtain expert weights. The combination weights of attributes are obtained using the correlation coefficient and the standard deviation (CCSD) method and the weight function in CPT.

Global existence of weak solutions and weak–strong uniqueness for nonisothermal Maxwell–Stefan systems *

The dynamics of multicomponent gas mixtures with vanishing barycentric velocity is described by Maxwell–Stefan equations with mass diffusion and heat conduction. The equations consist of the mass and energy balances, coupled to an algebraic system that relates the partial velocities and driving forces. The global existence of weak solutions to this system in a bounded domain with no-flux boundary conditions is proved by using the boundedness-by-entropy method. A priori estimates are obtained from the entropy inequality which originates from the consistent thermodynamic modelling. Furthermore, a conditional weak–strong uniqueness property is shown by using the relative entropy method.

The Separability Problem in Molecular Quantum Systems: Information-Theoretic Framework for Atoms in Molecules.

Even though molecules are fundamentally quantum entities, the concept of a molecule retains certain classical attributes concerning its constituents. This includes the empirical separability of a molecule into its three-dimensional, rigid structure in Euclidean space, a framework often obtained through experimental methods like X-Ray crystallography. In this work, we delve into the mathematical implications of partitioning a molecule into its constituent parts using the widely recognized Atoms-In-Molecules (AIM) schemes, aiming to establish their validity within the framework of Information Theory concepts. We have uncovered information-theoretical justifications for employing some of the most prevalent AIM schemes in the field of Chemistry, including Hirshfeld (stockholder partitioning), Bader's (topological dissection), and the quantum approach (Hilbert's space definition). In the first approach we have applied the generalized principle of minimum relative entropy derived from the Sharma-Mittal two-parameter functional, avoiding the need for an arbitrary selection of reference promolecular atoms. Within the ambit of topological-information partitioning, we have demonstrated that the Fisher information of Bader's atoms conform to a comprehensive theory based on the Principle of Extreme Physical Information avoiding the need of employing the Schwinger's principle, which has been proven to be problematic. For the quantum approach we have presented information-theoretic justifications for conducting Löwdin symmetric transformations on the density matrix to form atomic Hilbert spaces generating orthonormal atomic orbitals with maximum occupancy for a given wavefunction.

Diagnostics of Mixed-State Topological Order and Breakdown of Quantum Memory

Topological quantum memory can protect information against local errors up to finite error thresholds. Such thresholds are usually determined based on the success of decoding algorithms rather than the intrinsic properties of the mixed states describing corrupted memories. Here we provide an intrinsic characterization of the breakdown of topological quantum memory, which both gives a bound on the performance of decoding algorithms and provides examples of topologically distinct mixed states. We employ three information-theoretical quantities that can be regarded as generalizations of the diagnostics of ground-state topological order, and serve as a definition for topological order in error-corrupted mixed states. We consider the topological contribution to entanglement negativity and two other metrics based on quantum relative entropy and coherent information. In the concrete example of the two-dimensional (2D) Toric code with local bit-flip and phase errors, we map three quantities to observables in 2D classical spin models and analytically show they all undergo a transition at the same error threshold. This threshold is an upper bound on that achieved in any decoding algorithm and is indeed saturated by that in the optimal decoding algorithm for the Toric code. Published by the American Physical Society 2024

Matrix trace inequalities related to the Tsallis relative entropies of real order, II

In this paper, as a continuation of two previous papers, we show matrix trace inequalities related to two Tsallis relative entropies of all real order: For density matrices ρ and σ, and each α∈R﹨{0}, the Tsallis relative entropy Dα(ρ|σ) is defined by Dα(ρ|σ)=−Tr(ρ1−ασα−ρα) and NTα(ρ|σ) is defined by NTα(ρ|σ)=−Tr[ρ♮ασ−ρα], where ♮α is defined by ρ♮ασ=ρ1/2(ρ−1/2σρ−1/2)αρ1/2. Then we show the order relation between two Tsallis relative entropies Dα and NTα of all real order α∈R and a 1-parameter extension of the path connecting them of all real order. Moreover, we show estimates of the difference between two Tsallis relative entropies of all real order.

Subsystem eigenstate thermalization hypothesis for translation invariant systems.

The eigenstate thermalization hypothesis for translation invariant quantum spin systems has been proved recently by using random matrices. In this paper, we study the subsystem version of the eigenstate thermalization hypothesis for translation invariant quantum systems without referring to random matrices. We first find a relation between the quantum variance and the Belavkin-Staszewski relative entropy. Then, by showing the small upper bounds on the quantum variance and the Belavkin-Staszewski relative entropy, we prove the subsystem eigenstate thermalization hypothesis for translation invariant quantum systems with an algebraic speed of convergence in an elementary way. The proof holds for most of the translation invariant quantum lattice models with exponential or algebraic decays of correlations.

Uncertainty relation for symmetric Petz-Rényi relative entropy.

Holevo introduced a fidelity between quantum states that is symmetric and as effective as the trace distance in evaluating their similarity. This fidelity is bounded by a function of the trace distance, a relationship to which we will refer as Holevo's inequality. More broadly, Holevo's fidelity is part of a one-parameter family of symmetric Petz-Rényi relative entropies, which in turn satisfy a Pinsker's-like inequality with respect to the trace distance. Although Holevo's inequality is tight, Pinsker's inequality is loose for this family. We show that the symmetric Petz-Rényi relative entropies satisfy a tight inequality with respect to the trace distance, improving Pinsker's and reproducing Holevo's as a specific case. Additionally, we show how this result emerges from a symmetric Petz-Rényi uncertainty relation, a result that encompasses several relations in quantum and stochastic thermodynamics.

Relative Entropy Gradient Sampler for Unnormalized Distribution

We propose a relative entropy gradient sampler (REGS) for sampling from unnormalized distributions. REGS is a particle method that seeks a sequence of simple nonlinear transforms iteratively pushing the initial samples from a reference distribution into the samples from an unnormalized target distribution. To determine the nonlinear transforms at each iteration, we consider the Wasserstein gradient flow of relative entropy. This gradient flow determines a path of probability distributions that interpolates the reference distribution and the target distribution. It is characterized by an ordinary differential equation (ODE) system with velocity fields depending on the density ratios of the density of evolving particles and the unnormalized target density. To sample with REGS, we need to estimate the density ratios and simulate the ODE system with particle evolution. We propose a novel nonparametric approach to estimating the logarithmic density ratio using neural networks. Extensive simulation studies on challenging multimodal 1D and 2D mixture distributions and Bayesian logistic regression on real datasets demonstrate that REGS has reasonable performance compared with popular samplers based on Wasserstein gradient flows. Supplementary materials for this article are available online.

Quantum Mechanics Based on an Extended Least Action Principle and Information Metrics of Vacuum Fluctuations

We show that the formulations of non-relativistic quantum mechanics can be derived from an extended least action principle. The principle can be considered as an extension of the least action principle from classical mechanics by factoring in two assumptions. First, the Planck constant defines the minimal amount of action a physical system needs to exhibit during its dynamics in order to be observable. Second, there is constant vacuum fluctuation along a classical trajectory. A novel method is introduced to define the information metrics to measure additional observability due to vacuum fluctuations, which is then converted to an additional action through the first assumption. Applying the variational principle to minimize the total actions allows us to recover the basic quantum formulations including the uncertainty relation and the Schrödinger equation in the position representation. In the momentum representation, the same method can be applied to obtain the Schrödinger equation for a free particle while further investigation is still needed for a particle with an external potential. Furthermore, the principle brings in new results on two fronts. At the conceptual level, we find that the information metrics for vacuum fluctuations are responsible for the origin of the Bohm quantum potential. Even though the Bohm potential for a bipartite system is inseparable, the underlying vacuum fluctuations are local. Thus, inseparability of the Bohm potential does not justify a non-local causal relation between the two subsystems. At the mathematical level, quantifying the information metrics for vacuum fluctuations using more general definitions of relative entropy results in a generalized Schrödinger equation that depends on the order of relative entropy. The extended least action principle is a new mathematical tool. It can be applied to derive other quantum formalisms such as quantum scalar field theory.

Diffuse-interface approximation and weak–strong uniqueness of anisotropic mean curvature flow

Abstract The purpose of this paper is to derive anisotropic mean curvature flow as the limit of the anisotropic Allen–Cahn equation. We rely on distributional solution concepts for both the diffuse and sharp interface models and prove convergence using relative entropy methods, which have recently proven to be a powerful tool in interface evolution problems. With the same relative entropy, we prove a weak–strong uniqueness result, which relies on the construction of gradient flow calibrations for our anisotropic energy functionals.

Resource-efficient quantum principal component analysis

Principal component analysis (PCA) is an important dimensionality reduction method in machine learning and data analysis. Recently, the quantum version of PCA has been established to diagonalize quantum states. Although these quantum algorithms promise quantum advantages, they require substantial resources beyond the reach of state-of-the-art quantum technologies. This work aims to reduce resource requirements and improve the efficiency of quantum PCA. Assuming that the quantum state is accessed through a purified quantum query model and a sampling model, we propose quantum algorithms that use minimal resource requirements for ancillary qubits to reveal properties of eigenvectors and eigenvalues of a state. In particular, we show that estimating eigenvalue λ with error ε and success probability larger than λ(1−η) requests a query complexity O~(ϵ−1) and a sample complexity O~(ϵ−2η−1) , respectively. To our knowledge, our result is the first quantum speedup that achieves asymptotic linear scaling in 1/ϵ for quantum PCA. As applications, we discuss estimating the minimum relative entropy of entanglement of bipartite pure-states and performing quantum state discrimination tasks. We show that quantum speedups are maintained when the pure state has a low Schmidt number and states of discrimination have a low rank. This study opens up a new quantum PCA method for high-dimensional quantum data analysis and discusses its application in quantum information processing tasks.

Exploring Entropy-Based Portfolio Strategies: Empirical Analysis and Cryptocurrency Impact

Exploring Entropy-Based Portfolio Strategies: Empirical Analysis and Cryptocurrency Impact

Geospatial Assessment of Urban Sprawl: A Case Study of Herat City, Afghanistan

This study aims to investigate the spatial and temporal dynamics of urban sprawl in Herat City, Afghanistan, from 2000 to 2021 using GIS and remote sensing data (Landsat 7 and 8). In this study, three machine learning algorithms, namely Support Vector Machine (SVM), Random Forest (RF), and Classification and Regression Trees (CART), were employed to classify the study area, and the accuracy of each algorithm for each study period was assessed. Based on the assessment results, the RF algorithm demonstrated higher accuracy and was selected as the classification algorithm. The Google Earth Engine cloud platform was utilized to classify the study area, and the GIS environment was employed for the creation of thematic layers. The analysis revealed a 30.06% increase in built-up areas from 2000 to 2021. Conversely, vegetation, water bodies, and bare land decreased by 8.51%, 1.08%, and 20.53%, respectively, during the same period. The findings indicated that Herat City experienced high-speed expansion between 2000 and 2013, while from 2013 to 2021; it developed at a medium speed. The Relative Shannon's entropy statistical algorithm was employed to quantify urban sprawl, and the results suggest a dispersed urban sprawl pattern. Internal migration to major cities due to conflicts, limited employment opportunities, and inadequate living amenities in rural areas has been a primary driver of urban sprawl in Herat City, Afghanistan.

Utility of intracranial EEG networks depends on re-referencing and connectivity choice.

Studies of intracranial EEG networks have been used to reveal seizure generators in patients with drug-resistant epilepsy. Intracranial EEG is implanted to capture the epileptic network, the collection of brain tissue that forms a substrate for seizures to start and spread. Interictal intracranial EEG measures brain activity at baseline, and networks computed during this state can reveal aberrant brain tissue without requiring seizure recordings. Intracranial EEG network analyses require choosing a reference and applying statistical measures of functional connectivity. Approaches to these technical choices vary widely across studies, and the impact of these technical choices on downstream analyses is poorly understood. Our objective was to examine the effects of different re-referencing and connectivity approaches on connectivity results and on the ability to lateralize the seizure onset zone in patients with drug-resistant epilepsy. We applied 48 pre-processing pipelines to a cohort of 125 patients with drug-resistant epilepsy recorded with interictal intracranial EEG across two epilepsy centres to generate intracranial EEG functional connectivity networks. Twenty-four functional connectivity measures across time and frequency domains were applied in combination with common average re-referencing or bipolar re-referencing. We applied an unsupervised clustering algorithm to identify groups of pre-processing pipelines. We subjected each pre-processing approach to three quality tests: (i) the introduction of spurious correlations; (ii) robustness to incomplete spatial sampling; and (iii) the ability to lateralize the clinician-defined seizure onset zone. Three groups of similar pre-processing pipelines emerged: common average re-referencing pipelines, bipolar re-referencing pipelines and relative entropy-based connectivity pipelines. Relative entropy and common average re-referencing networks were more robust to incomplete electrode sampling than bipolar re-referencing and other connectivity methods (Friedman test, Dunn-Šidák test P < 0.0001). Bipolar re-referencing reduced spurious correlations at non-adjacent channels better than common average re-referencing (Δ mean from machine ref = -0.36 versus -0.22) and worse in adjacent channels (Δ mean from machine ref = -0.14 versus -0.40). Relative entropy-based network measures lateralized the seizure onset hemisphere better than other measures in patients with temporal lobe epilepsy (Benjamini-Hochberg-corrected P < 0.05, Cohen's d: 0.60-0.76). Finally, we present an interface where users can rapidly evaluate intracranial EEG pre-processing choices to select the optimal pre-processing methods tailored to specific research questions. The choice of pre-processing methods affects downstream network analyses. Choosing a single method among highly correlated approaches can reduce redundancy in processing. Relative entropy outperforms other connectivity methods in multiple quality tests. We present a method and interface for researchers to optimize their pre-processing methods for deriving intracranial EEG brain networks.