Framework Of Information Theory Research Articles

Time-varying information measures: an adaptive estimation of information storage with application to brain-heart interactions.

Network Physiology is a rapidly growing field of study that aims to understand how physiological systems interact to maintain health. Within the information theory framework the information storage (IS) allows to measure the regularity and predictability of a dynamic process under stationarity assumption. However, this assumption does not allow to track over time the transient pathways occurring in the dynamical activity of a physiological system. To address this limitation, we propose a time-varying approach based on the recursive least squares algorithm (RLS) for estimating IS at each time instant, in non-stationary conditions. We tested this approach in simulated time-varying dynamics and in the analysis of electroencephalographic (EEG) signals recorded from healthy volunteers and timed with the heartbeat to investigate brain-heart interactions. In simulations, we show that the proposed approach allows to track both abrupt and slow changes in the information stored in a physiological system. These changes are reflected in its evolution and variability over time. The analysis of brain-heart interactions reveals marked differences across the cardiac cycle phases of the variability of the time-varying IS. On the other hand, the average IS values exhibit a weak modulation over parieto-occiptal areas of the scalp. Our study highlights the importance of developing more advanced methods for measuring IS that account for non-stationarity in physiological systems. The proposed time-varying approach based on RLS represents a useful tool for identifying spatio-temporal dynamics within the neurocardiac system and can contribute to the understanding of brain-heart interactions.

Contribution of information about acute and geriatric characteristics to decisions about life-sustaining treatment for old patients in intensive care

BackgroundLife-sustaining treatment (LST) in the intensive care unit (ICU) is withheld or withdrawn when there is no reasonable expectation of beneficial outcome. This is especially relevant in old patients where further functional decline might be detrimental for the self-perceived quality of life. However, there still is substantial uncertainty involved in decisions about LST. We used the framework of information theory to assess that uncertainty by measuring information processed during decision-making.MethodsDatasets from two multicentre studies (VIP1, VIP2) with a total of 7488 ICU patients aged 80 years or older were analysed concerning the contribution of information about the acute illness, age, gender, frailty and other geriatric characteristics to decisions about LST. The role of these characteristics in the decision-making process was quantified by the entropy of likelihood distributions and the Kullback–Leibler divergence with regard to withholding or withdrawing decisions.ResultsDecisions to withhold or withdraw LST were made in 2186 and 1110 patients, respectively. Both in VIP1 and VIP2, information about the acute illness had the lowest entropy and largest Kullback–Leibler divergence with respect to decisions about withdrawing LST. Age, gender and geriatric characteristics contributed to that decision only to a smaller degree.ConclusionsInformation about the severity of the acute illness and, thereby, short-term prognosis dominated decisions about LST in old ICU patients. The smaller contribution of geriatric features suggests persistent uncertainty about the importance of functional outcome. There still remains a gap to fully explain decision-making about LST and further research involving contextual information is required.Trial registration: VIP1 study: NCT03134807 (1 May 2017), VIP2 study: NCT03370692 (12 December 2017).

The maximum entropy principle in search theory

The paper considers the relationship between search theory and information theory. The traditional problem of search theory is to develop a search plan for a physical object in the sea or on land. The search plan has to develop the distribution of available search resources in such a way that the probability of detection the search object is to be maximum. The optimal solution is traditionally considered as so-called "uniformly optimal search plan", which provides a uniform distribution of the posterior probability of the location of the object as the search is conducted. At the same time, optimality simultaneously according to the criteria of maximum detection probability and equality of a posteriori probability is possible only for the exponential detection function, which is used most often in search theory. For other kinds of detection functions, the optimal solutions according to the specified criteria do not match. In this paper, the approach to this problem is considered on the basis of the maximum entropy principle. For a situation of discrete distribution, it is shown that, within the framework of information theory, the search problem has a simpler solution that does not depend on the kind of the detection function.

Flickering Emergences: The Question of Locality in Information-Theoretic Approaches to Emergence.

"Emergence", the phenomenon where a complex system displays properties, behaviours, or dynamics not trivially reducible to its constituent elements, is one of the defining properties of complex systems. Recently, there has been a concerted effort to formally define emergence using the mathematical framework of information theory, which proposes that emergence can be understood in terms of how the states of wholes and parts collectively disclose information about the system's collective future. In this paper, we show how a common, foundational component of information-theoretic approaches to emergence implies an inherent instability to emergent properties, which we call flickering emergence. A system may, on average, display a meaningful emergent property (be it an informative coarse-graining, or higher-order synergy), but for particular configurations, that emergent property falls apart and becomes misinformative. We show existence proofs that flickering emergence occurs in two different frameworks (one based on coarse-graining and another based on multivariate information decomposition) and argue that any approach based on temporal mutual information will display it. Finally, we argue that flickering emergence should not be a disqualifying property of any model of emergence, but that it should be accounted for when attempting to theorize about how emergence relates to practical models of the natural world.

The Dark Side of the Entangled Informational Universe

After having explored some basic theoretical concepts about the quantum information approach, we focus on Entropic Information Theory which is an informational approach mathematically based on the mass of the bit of information; massbit = \(\frac{kTIn(2)}{c^2}\). The mass of the bit of information and the new entropy formulae associated to it,S = k2\(\frac{TIn(2)t}{h}\) , and its alternative writings lead to new formulation,SBH = K\(\frac{c^3In(2)tevap}{16\pi^2GM}\), to calculate the entropy of black holes independently of the law of area. Being able to express the fine-grained gravitational entropy of a black hole using the rules of gravity, we can, at this level, speak of quantum gravity as emerging through the fundamentality of entangled quantum information by considering that information emerges from degree of freedom; indeed, information being a quantum state change due to the modification of one degree of freedom from the considered quantum system. In addition, we calculated the informational content of the observable universe using the entropic information formula, to obtain, 1.57 1099 bits, a result remarkably close to some previous estimates to account for all the dark matter missing in the visible Universe. After that, we calculated the amount of energy associated with this informational content using Landauer's principle, to obtain, 3.50 1076 Joules, a result that we can relate to dark energy estimates. Moreover, some deep considerations based on the perspectives of Entropic Information Theory have been explored. This new complete mathematical framework of Entropic Information Theory can explain various processes being several aspects of the same, entangled information, by considering that information emerges from degree of freedom, it is the theoretical framework of the entangled informational universe.

Entropic comparison of Landau-Zener and Demkov interactions in the phase space of a quadrupole billiard.

We investigate two types of avoided crossings in a chaotic billiard within the framework of information theory. The Shannon entropy in the phase space for the Landau-Zener interaction increases as the center of the avoided crossing is approached, whereas for the Demkov interaction, the Shannon entropy decreases as the center of avoided crossing is passed by with an increase in the deformation parameter. This feature can provide a new indicator for scar formation. In addition, it is found that the Fisher information of the Landau-Zener interaction is significantly larger than that of the Demkov interaction.

Quantum telescopy clock games

We consider the clock game-a task formulated in the framework of quantum information theory-that can be used to improve the existing schemes of quantum-enhanced telescopy. The problem of learning when a stellar photon reaches a telescope is translated into an abstract game, which we call the clock game. A winning strategy is provided that involves performing a quantum non-demolition measurement that verifies which stellar spatio-temporal modes are occupied by a photon without disturbing the phase information. We prove tight lower bounds on the entanglement cost needed to win the clock game, with the amount of necessary entangled bits equaling the number of time-bins being distinguished. This lower bound on the entanglement cost applies to any telescopy protocol that aims to non-destructively extract the time-bin information of an incident photon through local measurements, and our result implies that the protocol of Khabiboulline et al. [\text{Phys. Rev. Lett.} 123, 70504 (2019)] is optimal in terms of entanglement consumption. The full task of the phase extraction is also considered, and we show that the quantum Fisher information of the stellar phase can be achieved by local measurements and shared entanglement without the necessity of nonlinear optical operations. The optimal phase measurement is achieved asymptotically with increasing number of ancilla qubits, whereas a single qubit pair is required if nonlinear operations are allowed.

Second law of information dynamics

One of the most powerful laws in physics is the second law of thermodynamics, which states that the entropy of any system remains constant or increases over time. In fact, the second law is applicable to the evolution of the entire universe and Clausius stated, “The entropy of the universe tends to a maximum.” Here, we examine the time evolution of information systems, defined as physical systems containing information states within Shannon’s information theory framework. Our observations allow the introduction of the second law of information dynamics (infodynamics). Using two different information systems, digital data storage and a biological RNA genome, we demonstrate that the second law of infodynamics requires the information entropy to remain constant or to decrease over time. This is exactly the opposite to the evolution of the physical entropy, as dictated by the second law of thermodynamics. The surprising result obtained here has massive implications for future developments in genomic research, evolutionary biology, computing, big data, physics, and cosmology.

Unruh Effect and Information Entropy Approach

The Unruh effect can be considered a source of particle production. The idea has been widely employed in order to explain multiparticle production in hadronic and heavy-ion collisions at ultrarelativistic energies. The attractive feature of the application of the Unruh effect as a possible mechanism of the multiparticle production is the thermalized spectra of newly produced particles. In the present paper, the total entropy generated by the Unruh effect is calculated within the framework of information theory. In contrast to previous studies, here the calculations are conducted for the finite time of existence of the non-inertial reference frame. In this case, only a finite number of particles are produced. The dependence on the mass of the emitted particles is taken into account. Analytic expression for the entropy of radiated boson and fermion spectra is derived. We study also its asymptotics corresponding to low- and high-acceleration limiting cases. The obtained results can be further generalized to other intrinsic degrees of freedom of the emitted particles, such as spin and electric charge.

Spatially-dependent material uncertainties in anisotropic nonlinear elasticity: Stochastic modeling, identification, and propagation

This paper develops a stochastic model for the spatially-dependent material parameters parameterizing anisotropic strain energy density functions. The construction is cast within the framework of information theory, which is invoked to derive a least-informative model while ensuring consistency with theoretical requirements in finite elasticity. Specifically, almost sure polyconvexity and uniform growth conditions are enforced through proper repulsion constraints and regularization, hence making the forward problem of uncertainty propagation well posed. In addition, transformations arising from the linearization procedure are introduced for consistency and induce statistical dependencies in the primary variables. The latter include material moduli, a weight balancing between the isotropic and anisotropic contributions, and the angle defining the structural tensors. The identification of the model is subsequently performed, using an existing database on human arterial walls. Maximum likelihood estimators are obtained and provided for the adventitia, media, and intima layers, which enables the use of the proposed model as a generative surrogate for, e.g., training and classification in data-driven approaches integrating inter-patient variability. Finally, uncertainty propagation on a realistic, patient-specific geometry is conducted to demonstrate the efficiency of the stochastic modeling framework.

An efficient hybrid hash based privacy amplification algorithm for quantum key distribution

Privacy amplification (PA) is an essential part in a quantum key distribution (QKD) system, distilling a highly secure key from a partially secure string by public negotiation between two parties. The optimization objectives of privacy amplification for QKD are large block size, high throughput and low cost. For the global optimization of these objectives, a novel privacy amplification algorithm is proposed in this paper by combining multilinear-modular-hashing and modular arithmetic hashing. This paper proves the security of this hybrid hashing PA algorithm within the framework of both information theory and composition security theory. A scheme based on this algorithm is implemented and evaluated on a CPU platform. The results on a typical CV-QKD system indicate that the throughput of this scheme ( $$261\,\mathrm{Mbps}@2.6\times 10^8$$ input block size) is twice higher than the best existing scheme ( $$140\,\mathrm{Mbps}@1\times 10^8$$ input block size). Moreover, this scheme is implemented on a mobile CPU platform instead of a desktop CPU or a server CPU, which means that this algorithm has a better performance with a much lower cost and power consumption.

Información Mutua: Una forma de cuantificar las correlaciones

Within the framework of Information Theory, the existence of correlations between two random variables means that we can obtain information about one of them, just by measuring or observing the other random variable. In certain cases, this kind of relationship allows obtaining information about a variable even when the other is separated by a very large distance, that is, the process of obtaining information is non-local, an example (if not the only) is the quantum entanglement. These features of correlations make it interesting and important to study, classify and quantify them. The correlations are classified into classical correlations and quantum correlations, in addition they are quantified through the mutual information. Here we will present a natural way to define classical mutual information and then we will generalize it to the quantum case. Furthermore, every term in the definitions of mutual information will be interpreted using the concepts of classical and quantum entropy.

Bayesian track-before-detect algorithm for nonstationary sea clutter

Radar detection of small targets in sea clutter is a particularly demanding task because of the nonstationary characteristic of sea clutter. The track-before-detect (TBD) filter is an effective way to increase the signal-to-clutter ratio (SCR), thus improving the detection performance of small targets in sea clutter. To cope with the nonstationary characteristic of sea clutter, an easily-implemented Bayesian TBD filter with adaptive detection threshold is proposed and a new parameter estimation method is devised which is integrated into the detection process. The detection threshold is set according to the parameter estimation result under the framework of information theory. For detection of closely spaced targets, those within the same range cell as the one under test are treated as contribution to sea clutter, and a successive elimination method is adopted to detect them. Simulation results prove the effectiveness of the proposed algorithm in detecting small targets in nonstationary sea clutter, especially closely spaced ones.

Sensor placement for model identification of multi-story buildings under unknown earthquake ground motion

The number of sensors and their position is very important for the acquisition of high-quality measurement data in structural health monitoring (SHM), where the detection and evaluation of structural health status under unknown seismic ground motion have been an important research area. It is undoubtedly that the identification of structural model parameters under unknown seismic ground motion or joint identification of model parameters and seismic time-history input depends on the effective placement of sensors on structures. At present, the investigation of sensor configuration is mainly intended for structural model or modal parameter identification without taking into account the seismic effect, which cannot guarantee the effective contribution of the dynamic measurement data to the joint identification of ground motion and model parameters. To the best of authors knowledge, the optimal sensor configuration for the purpose of model parameter identification under unknown seismic ground motion has not yet been reported in the current literature. In the framework of Bayes'theorem and information theory, this paper proposes an optimal sensor placement methodology targeting at model parameter identification under unknown seismic ground motion. Based on the joint identification of ground motion and structural model parameters, the optimal configuration scheme for a given number of sensors is determined by maximizing a proposed expected entropy measure formed by utilizing the marginal distribution of model parameters prior to the availability of the measurement data. In addition, to solve the combinatorial optimization problem arising from the optimal sensor configuration, this paper also proposes a simple yet efficient strategy for revising the fitness function of genetic algorithm, which converts the continuous variable representation of the sensor configuration vector into a binary string representation. This completely avoids the common approaches relying on modifying various genetic operators to ensure the mandatory constraint of always configuring a fixed number of sensors. The validity of the proposed methodology is fully validated by numerical and experimental case studies conducted respectively for a 20-story shear-building model and a laboratory 6-story shear-building model under shaking table test.

The physical origin of a photon-number parity effect in cavity quantum electrodynamics

The rapidly increasing capability to modulate the physicochemical properties of atomic groups and molecules by means of their coupling to radiation, as well as the revolutionary potential of quantum computing for materials simulation and prediction, fuel the interest for non-classical phenomena produced by atom-radiation interaction in confined space. One of such phenomena is a “parity effect” that arises in the dynamics of an atom coupled to two degenerate cavity field modes by two-photon processes and manifests itself as a strong dependence of the field dynamics on the parity of the initial number of photons. Here we identify the physical origin of this effect in the quantum correlations that produce entanglement among the system components, explaining why the system evolution depends critically on the parity of the total number of photons. Understanding the physical underpinnings of the effect also allows us to characterize it within the framework of quantum information theory and to generalize it. Since a single photon addition/removal has dramatic effects on the system behavior, this effect may be usefully applied, also for amplification purposes, to optoelectronics and quantum information processing.

Lipidomics of Environmental Microbial Communities. II: Characterization Using Molecular Networking and Information Theory.

Structurally diverse, specialized lipids are crucial components of microbial membranes and other organelles and play essential roles in ecological functioning. The detection of such lipids in the environment can reveal not only the occurrence of specific microbes but also the physicochemical conditions to which they are adapted to. Traditionally, liquid chromatography coupled with mass spectrometry allowed for the detection of lipids based on chromatographic separation and individual peak identification, resulting in a limited data acquisition and targeting of certain lipid groups. Here, we explored a comprehensive profiling of microbial lipids throughout the water column of a marine euxinic basin (Black Sea) using ultra high-pressure liquid chromatography coupled with high-resolution tandem mass spectrometry (UHPLC-HRMS/MS). An information theory framework combined with molecular networking based on the similarity of the mass spectra of lipids enabled us to capture lipidomic diversity and specificity in the environment, identify novel lipids, differentiate microbial sources within a lipid group, and discover potential biomarkers for biogeochemical processes. The workflow presented here allows microbial ecologists and biogeochemists to process quickly and efficiently vast amounts of lipidome data to understand microbial lipids characteristics in ecosystems.

Ten key ICT challenges in the post-Shannon era

Considering the vision of building a fully connected, intelligent world, this paper investigates the technological challenges and proposes ten key challenges that the information and communications technologies (ICT) industry is facing from the perspectives of the generation, transmission, and processing of information. These challenges cover the subjects of information theory, operational research, control theory, computational theory, and artificial intelligence. The paper first discusses the backgrounds of these ten challenges and their definitions, and offers the literature reviews of the current research. Then, it presents the potential methods for solving these problems as well as the future research directions. Finally, the paper introduces the opportunities for making theoretical breakthroughs in order to solve these problems, including the following: Transcending the Shannon information theory to build a framework of semantic information theory; building a theoretical framework to improve the explainability of deep neural networks; overcoming the challenges of the traditional queuing theory and optimization theory related to solving networking problems, and developing new theories such as network queuing theory and network optimization theory; and overcoming the constraints of power consumption and complexity to build a theoretical framework for approximate computing.

Kähler fibrations in quantum information theory

We discuss the fibre bundle of co-adjoint orbits of compact Lie groups, and show how it admits a compatible Kähler structure. The case of the unitary group allows us to reformulate the geometric framework of quantum information theory. In particular, we show that the Fisher information tensor gives rise to a structure that is sufficiently close to a Kähler structure to generalise some classical result on co-adjoint orbits.

An overview of generalized entropic forms (a) (a)Contribution to the Focus Issue Progress on Statistical Physics and Complexity edited by Roberta Citro, Giorgio Kaniadakis, Claudio Guarcello, Antonio Maria Scarfone and Davide Valenti.

The aim of this focus article is to present a comprehensive classification of the main entropic forms introduced in the last fifty years in the framework of statistical physics and information theory. Most of them can be grouped into three families, characterized by two-deformation parameters, introduced respectively by Sharma, Taneja, and Mittal (entropies of degree )), by Sharma and Mittal (entropies of order ), and by Hanel and Thurner (entropies of class ). Many entropic forms examined will be characterized systematically by means of important concepts such as their axiomatic foundations à la Shannon-Khinchin and the consequent composability rule for statistically independent systems. Other critical aspects related to the Lesche stability of information measures and their consistency with the Shore-Johnson axioms will be briefly discussed on a general ground.

An information theory model for optimizing quantitative magnetic resonance imaging acquisitions

Acquisition parameter selection is currently performed empirically for many quantitative MRI (qMRI) acquisitions. Tuning parameters for different scan times, tissues, and resolutions requires some amount of trial and error. There is an opportunity to quantitatively optimize these acquisition parameters in order to minimize variability of quantitative maps and post-processing techniques such as synthetic image generation. The objective of this work is to introduce and evaluate a quantitative method for selecting parameters that minimize image variability. An information theory framework was developed for this purpose and applied to a 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) signal model for qMRI. In this framework, mutual information is used to measure the information gained by a measurement as a function of acquisition parameters, quantifying the information content of potential acquisitions and allowing informed parameter selection. The information theory framework was tested on artificial data generated from a representative mathematical phantom, measurements acquired on a qMRI multiparametric imaging standard phantom, and in vivo measurements in a human brain. The phantom measurements showed that higher mutual information calculated by the model correlated with smaller coefficient of variation in the reconstructed parametric maps, and in vivo measurements demonstrated that information-based calibration of acquisition parameters resulted in a decrease in parametric map variability consistent with model predictions.