Information-theoretic Measures Research Articles

Information theoretic measures on quantum droplets in ultracold atomic systems

We consider Shannon entropy, Fisher information, Rényi entropy, and Tsallis entropy to study the quantum droplet phase in Bose–Einstein condensates. In the beyond mean-field description, the Gross–Pitaevskii equation with Lee-Huang-Yang correction gives a family of quantum droplets with different chemical potentials. At a larger value of chemical potential, quantum droplet with sharp-top probability density distribution starts to form while it becomes flat top for a smaller value of chemical potential. We show that entropic measures can distinguish the shape change of the probability density distributions and thus can identify the onset of the droplet phase. During the onset of droplet phase, the Shannon entropy decreases gradually with the decrease of chemical potential and attains a minimum in the vicinity where a smooth transition from flat-top to sharp-top QDs occurs. At this stage, the Shannon entropy increases abruptly with the lowering of chemical potential. We observe an opposite trend in the case of Fisher information. These results are found to be consistent with the Rényi and Tsallis entropic measures.

Information theoretic measures for Lifshitz system

In this work, we have studied various mixed state information theoretic quantities for an excited state of Lifshitz spacetime in 3 + 1-dimensions. This geometry is the gravity dual to a class of 2 + 1-dimensional quantum field theories having Lifshitz symmetry. We have holographically calculated mutual information, entanglement wedge cross section, entanglement negativity and mutual complexity for strip like subsystems at the boundary. For this we have used the results of holographic entanglement entropy and complexity present in the literature. We first calculate all of these mentioned quantities for the pure state of Lifshitz spacetime. Then we have moved on to calculate all these quantities for excited state of the Lifshitz spacetime. The gravity dual of excited state of Lifshitz systems in field theory can be obtained by applying constant perturbations along the boundary direction. Further, we would like to mention that for the simplicity of calculation we are only considering results up to the first order in perturbation. The change in the obtained holographic information theoretic quantities are then related to entanglement entropy, entanglement pressure, entanglement chemical potential and charge using the stress tensor complex. These relations are analogous to the first law of entanglement thermodynamics given earlier in the literature. All the calculations are carried out for both values of dynamical scaling exponent (z) present in the Lifshitz field theory.

Bias in O-Information Estimation.

Higher-order relationships are a central concept in the science of complex systems. A popular method of attempting to estimate the higher-order relationships of synergy and redundancy from data is through the O-information. It is an information-theoretic measure composed of Shannon entropy terms that quantifies the balance between redundancy and synergy in a system. However, bias is not yet taken into account in the estimation of the O-information of discrete variables. In this paper, we explain where this bias comes from and explore it for fully synergistic, fully redundant, and fully independent simulated systems of n=3 variables. Specifically, we explore how the sample size and number of bins affect the bias in the O-information estimation. The main finding is that the O-information of independent systems is severely biased towards synergy if the sample size is smaller than the number of jointly possible observations. This could mean that triplets identified as highly synergistic may in fact be close to independent. A bias approximation based on the Miller-Maddow method is derived for the O-information. We find that for systems of n=3 variables the bias approximation can partially correct for the bias. However, simulations of fully independent systems are still required as null models to provide a benchmark of the bias of the O-information.

Non-Abelian Transport Distinguishes Three Usually Equivalent Notions of Entropy Production

We extend entropy production to a deeply quantum regime involving noncommuting conserved quantities. Consider a unitary transporting conserved quantities (“charges”) between two systems initialized in thermal states. Three common formulas model the entropy produced. They respectively cast entropy as an extensive thermodynamic variable, as an information-theoretic uncertainty measure, and as a quantifier of irreversibility. Often, the charges are assumed to commute with each other (e.g., energy and particle number). Yet quantum charges can fail to commute. Noncommutation invites generalizations, which we posit and justify, of the three formulas. The noncommutation of charges, we find, breaks the equivalence of the formulas. Furthermore, different formulas quantify different physical effects of the noncommutation of charges on entropy production. For instance, entropy production can signal contextuality—true nonclassicality—by becoming nonreal. This work opens up stochastic thermodynamics to noncommuting—and so particularly quantum—charges. Published by the American Physical Society 2024

Measuring and understanding information storage and transfer in a simulated human gut microbiome.

Considering biological systems as information processing entities and analyzing their organizational structure via information-theoretic measures has become an established approach in life sciences. We transfer this framework to a field of broad general interest, the human gut microbiome. We use BacArena, a software combining agent-based modelling and flux-balance analysis, to simulate a simplified human intestinal microbiome (SIHUMI). In a first step, we derive information theoretic measures from the simulated abundance data, and, in a second step, relate them to the metabolic processes underlying the abundance data. Our study provides further evidence on the role of active information storage as an indicator of unexpected structural change in the observed system. Besides, we show that information transfer reflects coherent behavior in the microbial community, both as a reaction to environmental changes and as a result of direct effective interaction. In this sense, purely abundance-based information theoretic measures can provide meaningful insight on metabolic interactions within bacterial communities. Furthermore, we shed light on the important however little noticed technical aspect of distinguishing immediate and delayed effects in the interpretation of local information theoretical measures.

Crown ether dopant to reduce ion suppression and improve detection in the liquid microjunction surface sampling probe.

Sodium and potassium are required in agar media for the growth of some microorganisms (e.g., marine bacteria). However, alkali cations are a significant source of contamination for mass spectrometry causing ion suppression and adduct formation. Conventionally, salts can be removed before mass spectrometric analysis with appropriate and often lengthy sample preparation. The direct mass spectrometric sampling of bacterial colonies grown on agar media seeks to minimize or eliminate sample preparation to improve workflow. However, this may exacerbate ion suppression and contamination since these metal cations will degrade spectral quality and limit the rapid profiling of microbial metabolites. Different approaches are needed to sequester sodium and potassium ions to minimize unwanted background interferences. Herein, we use crown ethers (CEs) in combination with a liquid microjunction surface sampling probe (LMJ-SSP) to directly sample the surface of the bacterial colonies from two marine bacteria species (Pseudoalteromonas rubra DSM6842 and Pseudoalteromonas tunicata DSM 14096). CEs (e.g., 18-crown-6 or 15-crown-5) are added to the carrier solvent of the LMJ-SSP, the chemical noise is reduced, and spectra are easier to interpret. The liquid microjunction formed at the tip of LMJ-SSP was used to directly touch bacterial colonies on agar. The carrier solvent was either methanol (100%) or methanol: H2O (50:49.9%) with or without 0.01% CEs. Information-theoretic measures are employed to investigate qualitative changes between spectra before and after adding CEs. Our work demonstrates the capability of CEs to reduce background interferences within the direct profiling of bacterial colonies from agar plates. The data obtained from both P. rubra DSM6842 and P. tunicata DSM 14096 show that CEs can be used to mitigate the salty background and improve compound detection. Our approach can be implemented in natural product discovery using LMJ-SSP to allow fast and accurate detection of interesting/novel compounds.

Polygon relations and subadditivity of entropic measures for discrete and continuous multipartite entanglement

In a recent work by us Ge et al [Phys. Rev. A 110, L010402 (2024)], we have derived a series of polygon relations of bipartite entanglement measures that is useful to reveal entanglement properties of discrete, continuous, and even hybrid multipartite quantum systems. In this work, with the information-theoretical measures of Rényi and Tsallis entropies, we study the relationship between the polygon relation and the subadditivity of entropy. In particular, the entropy-polygon relations are derived for pure multi-qubit states and then generalized to multi-mode Gaussian states, by utilizing the known results from the quantum marginal problem. Then the equivalence between the polygon relation and subadditivity is established, in the sense that for all discrete or continuous multipartite states, the polygon relation holds if and only if the underlying entropy is subadditive. As a byproduct, the subadditivity of Rényi and Tsallis entropies is proven for all bipartite Gaussian states. Finally, the difference between polygon relations and monogamy relations is clarified, and generalizations of our results are discussed. Our work provides a better understanding of the rich structure of multipartite states, and hence is expected to be helpful for the study of multipartite entanglement.

Incoherence: A Generalized Measure of Complexity to Quantify Ensemble Divergence in Multi-Trial Experiments and Simulations.

Complex systems pose significant challenges to traditional scientific and statistical methods due to their inherent unpredictability and resistance to simplification. Accurately detecting complex behavior and the uncertainty which comes with it is therefore essential. Using the context of previous studies, we introduce a new information-theoretic measure, termed "incoherence". By using an adapted Jensen-Shannon Divergence across an ensemble of outcomes, we quantify the aleatoric uncertainty of the system. First we compared this measure to established statistical tests using both continuous and discrete data. Before demonstrating how incoherence can be applied to identify key characteristics of complex systems, including sensitivity to initial conditions, criticality, and response to perturbations.

Information theoretic measures in one-dimensional Dunkl oscillator

Information theoretic measures in one-dimensional Dunkl oscillator

Information-theoretic measures and Compton profile of H atom under finite oscillator potential

Information-theoretic measures for nl (2L) states of a H atom (with n=1−10 and l=0−2 , where n and l denote principal and angular momentum quantum numbers) have been investigated within a quantum dot by utilizing the Ritz variational principle, with the help of a Slater-type basis set. A well-established two-parameter (depth and width) model of finite oscillator potential is used to simulate the dot environment. The variationally optimized position (r)-space wave function is utilized to determine the momentum (p)-space wave function, leading to the generation of p-space radial density distribution. We explore the impact of cavity parameters on quantum information theoretic measures, such as Shannon (S) and Fisher information (I) entropy, in the ground as well as the excited state. The results of S were also used to test the Bialynicki–Birula–Mycielski inequality, related to the entropic uncertainty principle for the confined H atom. Some simple new fitting laws pertaining to S and I have been proposed. Furthermore, the p-space radial density is employed to derive the Compton profile of the confined H atom. Possible tunability of S,I and Compton profiles with respect to the parameters is noted.

Information-theoretical limit on the estimates of dissipation by molecular machines using single-molecule fluorescence resonance energy transfer experiments.

Single-molecule fluorescence resonance energy transfer (FRET) experiments are commonly used to study the dynamics of molecular machines. While in vivo molecular processes often break time-reversal symmetry, the temporal directionality of cyclically operating molecular machines is often not evident from single-molecule FRET trajectories, especially in the most common two-color FRET studies. Solving a more quantitative problem of estimating the energy dissipation/entropy production by a molecular machine from single-molecule data is even more challenging. Here, we present a critical assessment of several practical methods of doing so, including Markov-model-based methods and a model-free approach based on an information-theoretical measure of entropy production that quantifies how (statistically) dissimilar observed photon sequences are from their time reverses. The Markov model approach is computationally feasible and may outperform model free approaches, but its performance strongly depends on how well the assumed model approximates the true microscopic dynamics. Markov models are also not guaranteed to give a lower bound on dissipation. Meanwhile, model-free, information-theoretical methods systematically underestimate entropy production at low photoemission rates, and long memory effects in the photon sequences make these methods demanding computationally. There is no clear winner among the approaches studied here, and all methods deserve to belong to a comprehensive data analysis toolkit.

Deep learning insights into non-universality in the halo mass function

ABSTRACT The abundance of dark matter haloes is a key cosmological probe in forthcoming galaxy surveys. The theoretical understanding of the halo mass function (HMF) is limited by our incomplete knowledge of the origin of non-universality and its cosmological parameter dependence. We present a deep-learning model which compresses the linear matter power spectrum into three independent factors which are necessary and sufficient to describe the $z=0$ HMF from the state-of-the-art Aemulus emulator to sub-per cent accuracy in a wCDM$+N_\mathrm{eff}$ parameter space. Additional information about growth history does not improve the accuracy of HMF predictions if the matter power spectrum is already provided as input, because required aspects of the former can be inferred from the latter. The three factors carry information about the universal and non-universal aspects of the HMF, which we interrogate via the information-theoretic measure of mutual information. We find that non-universality is captured by recent growth history after matter-dark-energy equality and $N_{\rm eff}$ for $M\sim 10^{13} \, \mathrm{M_\odot }\, h^{-1}$ haloes, and by $\Omega _{\rm m}$ for $M\sim 10^{15} \, \mathrm{M_\odot }\, h^{-1}$. The compact representation learnt by our model can inform the design of emulator training sets to achieve high emulator accuracy with fewer simulations.

Outsourcing Control Requires Control Complexity.

An embodied agent influences its environment and is influenced by it. We use the sensorimotor loop to model these interactions and quantify the information flows in the system by information-theoretic measures. This includes a measure for the interaction among the agent's body and its environment, often referred to as morphological computation. Additionally, we examine the controller complexity, which can be seen in the context of the integrated information theory of consciousness. Applying this framework to an experimental setting with simulated agents allows us to analyze the interaction between an agent and its environment, as well as the complexity of its controller. Previous research revealed that a morphology adapted well to a task can substantially reduce the required complexity of the controller. In this work, we observe that the agents first have to understand the relevant dynamics of the environment to interact well with their surroundings. Hence an increased controller complexity can facilitate a better interaction between an agent's body and its environment.

"Modeling Diffusive Search by Non-Adaptive Sperm: Empirical and Computational Insights".

During fertilization, mammalian sperm undergo a winnowing selection process that reduces the candidate pool of potential fertilizers from ~106-1011 cells to 101-102 cells (depending on the species). Classical sperm competition theory addresses the positive or 'stabilizing' selection that acts on sperm phenotypes within populations of organisms but does not strictly address the developmental consequences of sperm traits among individual organisms that are under purifying selection during fertilization. It is the latter that is of utmost concern for improving assisted reproductive technologies (ART) because 'low fitness' sperm may be inadvertently used for fertilization during interventions that rely heavily on artificial sperm selection, such as intracytoplasmic sperm injection (ICSI). Importantly, some form of sperm selection is used in nearly all forms of ART (e.g., differential centrifugation, swim-up, or hyaluronan binding assays, etc.). To date, there is no unifying quantitative framework (i.e., theory of sperm selection) that synthesizes causal mechanisms of selection with observed natural variation in individual sperm traits. In this report, we reframe the physiological function of sperm as a collective diffusive search process and develop multi-scale computational models to explore the causal dynamics that constrain sperm 'fitness' during fertilization. Several experimentally useful concepts are developed, including a probabilistic measure of sperm 'fitness' as well as an information theoretic measure of the magnitude of sperm selection, each of which are assessed under systematic increases in microenvironmental selective pressure acting on sperm motility patterns.

Distributed robust support vector ordinal regression under label noise

Ordinal regression (OR) methods are designed for a type of classification problems where data labels have natural orders. In practice, data may be corrupted by label noise, which affects the training process thus degrading the generalization performance of OR methods. In OR, data are usually assumed to have latent variables underlying the ordinal labels, and label noise exhibits a special characteristic that it usually causes large latent variable value deviations. However, there are few existing works on OR considering label noise, and the existing works do not utilize the above-mentioned characteristic. Besides, most of the existing OR methods are centralized, which are inapplicable in some realistic distributed applications. In this paper, we utilize the characteristic of label noise in OR to develop a distributed robust support vector ordinal regression method (dRSVOR) under label noise. Specifically, after analyzing the characteristic of label noise in OR, we take the form of SVOR with explicit constraints to achieve robustness to one type of mislabeled samples. Then, we adopt correntropy, an information-theoretic measure, to achieve robustness to the other type of mislabeled samples. Theoretically, we analyze the consensus and convergence of dRSVOR. Experimentally, we conduct experiments on both synthetic data and real OR datasets to illustrate the effectiveness of the proposed method. The results show that the centralized version of dRSVOR outperforms several state-of-the-art OR methods considering label noise in centralized circumstances with label noise, and dRSVOR could approach the performance of the centralized version despite additional constraints in distributed scenarios.

Information content and optimization of self-organized developmental systems

A key feature of many developmental systems is their ability to self-organize spatial patterns of functionally distinct cell fates. To ensure proper biological function, such patterns must be established reproducibly, by controlling and even harnessing intrinsic and extrinsic fluctuations. While the relevant molecular processes are increasingly well understood, we lack a principled framework to quantify the performance of such stochastic self-organizing systems. To that end, we introduce an information-theoretic measure for self-organized fate specification during embryonic development. We show that the proposed measure assesses the total information content of fate patterns and decomposes it into interpretable contributions corresponding to the positional and correlational information. By optimizing the proposed measure, our framework provides a normative theory for developmental circuits, which we demonstrate on lateral inhibition, cell type proportioning, and reaction-diffusion models of self-organization. This paves a way toward a classification of developmental systems based on a common information-theoretic language, thereby organizing the zoo of implicated chemical and mechanical signaling processes.

An information theoretic metric for measurement vulnerability to data integrity attacks on smart grids

AbstractA novel metric that describes the vulnerability of the measurements in power systems to data integrity attacks is proposed. The new metric, coined vulnerability index (VuIx), leverages information theoretic measures to assess the attack effect in terms of the fundamental limits of the disruption and detection tradeoff. The result of computing the VuIx of the measurements in the system yields an ordering of their vulnerability based on the degree of exposure to data integrity attacks. This new framework is used to assess the measurement vulnerability of IEEE 9‐bus and 30‐bus test systems and it is observed that power injection measurements are significantly more vulnerable to data integrity attacks than power flow measurements. A detailed numerical evaluation of the VuIx values for IEEE test systems is provided.

Large-scale patterns set the predictability limit of extreme events in Kolmogorov flow

Events of extreme intensity in turbulent flows from atmospheric to industrial scales have a strong social and economic impact, and hence there is a need to develop models and indicators which enable their early prediction. Part of the difficulty here stems from the intrinsic sensitivity to initial conditions of turbulent flows. Despite recent progress in understanding and predicting extreme events, the question of how far in advance they can be ideally predicted (without model error and subject only to uncertainty in the initial conditions) remains open. Here we study the predictability limit of extreme dissipation bursts in the two-dimensional Kolmogorov flow by applying information-theoretic measures to massive statistical ensembles with more than $10^7$ direct numerical simulations. We find that extreme events with similar intensity and structure can exhibit disparate predictability due to different causal origins. Specifically, we show that highly predictable extreme events evolve from distinct large-scale circulation patterns. We thus suggest that understanding all the possible routes to the formation of extreme events is necessary to assess their predictability.

Graphs with minimum degree-entropy

We continue studying extremal values of the degree-entropy, which is an information-theoretic measure defined as the Shannon entropy based on the information functional involving vertex degrees. For a graph with a given number of vertices and edges achieving the minimum entropy value, we show its unique structure. Also, a tight lower bound for the entropy in bipartite graphs with a given number of vertices and edges is proved. Our result directly derives the result of Cao et al. (2014) that for a tree with a given number of vertices, the minimum value of the entropy is attained if and only if the tree is the star.

Neural Joint Entropy Estimation.

Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics, and data compression. Over the years, a variety of estimation schemes have been suggested. However, despite significant progress, most methods still struggle when the sample is small, compared to the variable's alphabet size. In this work, we introduce a practical solution to this problem, which extends the work of McAllester and Statos. The proposed scheme uses the generalization abilities of cross-entropy estimation in deep neural networks (DNNs) to introduce improved entropy estimation accuracy. Furthermore, we introduce a family of estimators for related information-theoretic measures, such as conditional entropy and mutual information (MI). We show that these estimators are strongly consistent and demonstrate their performance in a variety of use cases. First, we consider large alphabet entropy estimation. Then, we extend the scope to MI estimation. Next, we apply the proposed scheme to conditional MI estimation, as we focus on independence testing tasks. Finally, we study a transfer entropy (TE) estimation problem. The proposed estimators demonstrate improved performance compared to existing methods in all of these setups.