Unconditional Entropy Research Articles

The use of entropy and information analysis to estimate the milk productivity of the Black-and-White dairy breed cows depending on their lineal affiliation

In modern realities, an important place for the effective management of the cattle breeding industry is a properly implemented selection and breeding process with the sampling of cows that are marked by the highest indicators of a set of productive qualities. In this case, the application of entropy and information analysis is one of the most expedient, as it makes it possible to assess the economic and useful qualities of animals as fully as possible. The purpose of the study is to evaluate and analyze the degree of organization of the biological system of dairy cattle productive traits under the influence of hereditary factors such as the age and origin of cows. During the research, methods generally accepted in zootechnics and methods using information and entropy analysis, which are adapted and modified in animal husbandry, were used. The data obtained from the entropy and information analysis show that for cattle of the Black-and-White breed of different lineal affiliations, the level of organization of systems varies – R from 0.009 to 1.341 bits. Moreover, the most stable trait from the point of view of variability was the fat content in milk, the level of unconditional entropy in the section of lactations was equal to 3.333-4.550 bits, which indicates a smaller influence of disorganized factors on the level of manifestation of this trait, and a greater dependence on hereditary factors, i.e. origin. Although in general, a reliable influence of the lineal affiliation of cows on indicators of unconditional entropy and organization of the system was not established, the influence of the age factor on indicators of entropy was observed. Thus, the researched livestock is not characterized by uniformity and consolidation in terms of the main selection characteristics, except for the fat content in milk, which indicates a wide range of variability and serves as a flexible material both for selection and breeding work and for increasing the level of milk productivity. Therefore, the use of empirical data of information theory can be a kind of marker when predicting hereditary traits of a particular productivity, since entropy and information analysis provides wider and deeper values of trait variability

Informational steady states and conditional entropy production in continuously monitored systems: The case of Gaussian systems

The act of measuring a system has profound consequences of dynamical and thermodynamic nature. In particular, the degree of irreversibility ensuing from a non-equilibrium process is strongly affected by measurements aimed at acquiring information on the state of a system of interest: the conditional and unconditional entropy production, which quantify the degree of irreversibility of the open system's dynamics, are related to each other by clearly interpreted informational quantities. Building on a recently proposed collisional-model framework [G. T. Landi {\it et al.}, arXiv:2103.06247], we investigate the case of continuous-variable information carriers prepared in Gaussian states and undergoing Gaussian processes. We build up a toolbox that fully characterizes the thermodynamics of continuously measured non-equilibrium Gaussian systems and processes, illustrating how the instruments hereby introduced provide key insight into recent experiments on mesoscopic quantum systems [Phys. Rev. Lett, {\bf 125}, 080601 (2020)].

Randomness Extraction via a Quantum Generalization of the Conditional Collision Entropy

Randomness extraction against side information is the art of distilling from a given source a key which is almost uniform conditioned on the side information. This paper provides randomness extraction against quantum side information whose extractable key length is given by a quantum generalization of the collision entropy, which is smoothed and conditioned differently from how this is done in existing schemes. Based on the fact that the collision entropy is not subadditive, its optimization with respect to additional side information is introduced, and is shown to be asymptotically optimal. The lower bound derived there for general states is expressed as the difference between two unconditional entropies and its evaluation reduces to an eigenvalue problem of two states, which are the entire state and the marginal state of side information.

Statistical estimation of conditional Shannon entropy

The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector ofdfactors having a density w.r.t. the Lebesgue measure in ℝd. Namely, the mixed-pair model (X,Y) is considered whereXandYtake values in ℝdand an arbitrary finite set, respectively. Such models include, for instance, the famous logistic regression. In contrast to the well-known Kozachenko–Leonenko estimates of unconditional entropy the proposed estimates are constructed by means of the certain spacial order statistics (ork-nearest neighbor statistics wherek=kndepends on amount of observationsn) and a random number of i.i.d. observations contained in the balls of specified random radii. The asymptotic unbiasedness andL2-consistency of the new estimates are established under simple conditions. The obtained results can be applied to the feature selection problem which is important,e.g., for medical and biological investigations.

Features of calculating the information entropy of the text in case of attacking the linguistic stegosystem by semantic compression

The article deals with the improvement of well-known methods for calculating the entropy of the text, and the description of the information entropy of the text calculating peculiarities in case of the semantic compression attack on the linguistic stegosystem, implemented in the cognominal program complex. The problem of determining the natural language text entropy in the context of further discursive analysis and semantic redundancy removal is formalized. Additional parameters that contribute to determining the semantic entropy of meaningful and artificially generated text for a semantic compression attack on the linguistic stegosystem, the container of which is textual information of natural (English) language are entered. The entropy variety for different language styles is substantiated and its changing according to the style is explained due to the need of adding specialized terminology dictionaries to the general terminology dictionary. In addition to the calculation features of conditional and unconditional entropy in case of using the software complex for attack the linguistic stegosystem, the dictionary size used in it and the prescribed grammar rules size are given, which are the additional parameters determining the entropy calculation in a particular case. The maximum entropy calculation for meaningless texts and the amount of information of a single word or a grammatical form calculation in case of maximum and real entropy are shown. In addition, the calculation of the semantic compression limit is given and the task of determining the semantic information redundancy is formalized. Thus, it becomes possible to determine the quality of the compression attack, carried out on the basis of the software complex use. The obtained results can be used in further research to improve the means of conducting an attack, which will increase its efficiency by maximally approximating the semantic compression limit.

Entropy stable modeling of non-isothermal multi-component diffuse-interface two-phase flows with realistic equations of state

In this paper, we consider mathematical modeling and numerical simulation of non-isothermal compressible multi-component diffuse-interface two-phase flows with realistic equations of state. A general model with the general reference velocity is derived rigorously through thermodynamical laws and Onsager’s reciprocal principle, and it is capable of characterizing compressibility and partial miscibility between multiple fluids. We prove a novel relation between the pressure, temperature and chemical potentials, which results in a new formulation of the momentum conservation equation indicating that the gradients of chemical potentials and temperature become the primary driving force of the fluid motion except for the external forces. A key challenge in numerical simulation is to develop entropy stable numerical schemes preserving the laws of thermodynamics. Based on the convex–concave splitting of Helmholtz free energy density with respect to molar densities and temperature, we propose an entropy stable numerical method, which solves the total energy balance equation directly, and thus, naturally satisfies the first law of thermodynamics. Unconditional entropy stability (the second law of thermodynamics) of the proposed method is proved by estimating the variations of Helmholtz free energy and kinetic energy with time steps. Numerical results validate the proposed method.

ANALYSIS OF INFORMATION FACTORS FOR DESIGNING INTELLECTUAL MECHATRONIC SYSTEM

The paper proposes to evaluate achievement of main results in operation of intellectual mechatronic systems with digital control by the obtained information effect. In this respect, common information requirements with intellectual components are considered as a basic information factor which influences on the process of mechatronic system designing. Therefore, some parameters have been accentuated and they can help to provide rather complete description of the processes used for obtaining and using systematic information within the volume of the intellectual mechatronic system. Conformity degree of control vector parameters synthesized by the system and identification results of its current states have been selected as an information criterion of the control efficiency. A set of expected probability values for location of each parameter of an control object and a mechatronic system within the required tolerances has been used for formation of possible states. The paper shows that when a complex information description of the system is used then it is expedient to use an expert assessment of selection probability for allowable control vectors which ensure a system transfer to favorable states. This approach has made it possible to pinpoint main information and technical specifications of the intellectual mechatronic system: structural construction (informational and technical compatibility and information matching of its components); control object (uncertainty of its state and information vector, information capacity of the mechatronic system); control actions (their hierarchy and entropic balance of control process, managerial resource of mechatronic system); functioning result (informational effect and control efficiency criterion, probabilistic selection of system states). In accordance with the fulfilled analysis it is possible to note the most effective directions for practical use of the proposed informational approach for creation of the intellectual mechatronic system: comparison of alternative design solutions based on the analysis of calculation assessments on unconditional entropy of the control object and the system; simulation with the aim to accept a final systematic option while constructing digital controlling block of the created system; complex application of experts’ knowledge, most comprehensive introduction of knowledge to the process of mechatronic system designing.

Hurst–Kolmogorov dynamics as a result of extremal entropy production

It is demonstrated that extremization of entropy production of stochastic representations of natural systems, performed at asymptotic times (zero or infinity) results in constant derivative of entropy in logarithmic time and, in turn, in Hurst–Kolmogorov processes. The constraints used include preservation of the mean, variance and lag-1 autocovariance at the observation time step, and an inequality relationship between conditional and unconditional entropy production, which is necessary to enable physical consistency. An example with real world data illustrates the plausibility of the findings.

Optimal odor intensity in olfactory neuronal models

Signal processing in olfactory systems is initiated by binding of odorant molecules to receptor molecules embedded in the membranes of sensory neurons. An approach, which we use here, is based on stochastic variant ofthe law of mass action as a neuronal model. A model experiment is considered, in which a fixed odorant concentration is applied several times and realizations of steady-state characteristics are observed. The response is assumed to be a random variable with some probability density function belonging to a parametric family with the signal as a parameter. As a measure how well the signal can be estimated from the response, the Fisher information and its lower bounds are used. Another optimality measures are based on the theory of information, especially conditional and unconditional differential entropy. The study extends our previous results.

Analysis of the factors influencing information transmission at the calyx of Held

The calyx of Held is a giant glutamatergic synapse located in the medial nucleus of the trapezoid body (MNTB) of the mammalian auditory brainstem. It is an important model system for studying short-term plasticity because each postsynaptic MNTB neuron receives only one giant synapse, and it is possible to record both pre- and postsynaptic events simultaneously. Physiological experiments show that the evoked excitatory postsynaptic current (EPSC) from the calyx displays a significant depression in response amplitude during a sustained stimulus train [1]. This observation is the result of interactions between various pre- and postsynaptic components occurring across multiple time-scales. Hundreds of readily releasable vesicles (RRVP) aggregate at the different active zones on the presynaptic terminal. The stochastic release of neurotransmitter from the RRVPs (triggered by influx of calcium ions during action potentials, APs), can exhaust the supply of vesicles. Simultaneously, the depleted RRVPs are constantly replenished by a large vesicle reserve pool, and this replenishment is also enhanced by a calcium-dependent process (mediated by the presynaptic APs). The amplitude of the presynaptic AP-evoked calcium ion concentration is affected by inactivation and facilitation of voltage-gated calcium channels as well as activation of presynaptic metabotropic glutamate receptors (mGluRs). In this study we extend our deterministic, multiple time-scale model of the calyx of Held [2,3] to a stochastic version. We use information theory to measure the amount of information transmitted between pre- and postsynaptic compartments [4]. A series of long, homogeneous Poisson spike trains with mean frequencies up to several hundred Hertz are used to stimulate the calyx model. This spike train is repeated many times, allowing the calculation of the conditional and unconditional entropy of the postsynaptic EPSC amplitude in response to presynaptic interspike intervals (ISI). The mutual information, a measure of the information content of the postsynaptic response (EPSC) about the afferent spike train (ISI), is then computed in terms of conditional and unconditional entropies. The results show that the information content in the postsynaptic response is influenced by the degree of variation of presynaptic calcium ion concentration. Rapid onset and recovery from facilitation maintains high information transfer rates across the frequency range. Suppression of calcium transients by slowly recovering inactivation and mGluR activation results in less information transmission, but prevents depletion of the RRVPs. Fast-acting postsynaptic receptor desensitisation also contributes to information transmission, but in a competitive way to facilitation.

Kolmogorov entropy in the context of computability theory

We consider the overgraph of the Kolmogorov entropy function and study whether it is a complete enumerable set with respect to different types of reductions. It turns out that (for any type of entropy) the overgraph of the conditional entropy function is m-complete, but the overgraph of the unconditional entropy function is not m-complete (and also not bT-complete). For tt-completeness, the situation is more subtle: the overgraph of the unconditional prefix entropy may be tt-complete or incomplete depending on the optimal programming system used in the definition of entropy. To prove these results we use the notion of r-separability and its effective version introduced in this article for the first time.

Predictability problems of global change as seen through natural systems complexity description. 2. Approach

Developing the general statements of the proposed global change theory, outlined in Part 1 of the publication, Kolmogorov′s probability space is used to study properties of information measures (unconditional, joint and conditional entropies, information divergence, mutual information, etc.). Sets of elementary events, the specified algebra of their sub‐sets and probability measures for the algebra are composite parts of the space. The information measures are analyzed using the mathematical expectance operator and the adequacy between an additive function of sets and their equivalents in the form of the measures. As a result, explanations are given to multispectral satellite imagery visualization procedures using Markov′s chains of random variables represented by pixels of the imagery. The proposed formalism of the information measures application enables to describe the natural targets complexity by syntactically governing probabilities. Asserted as that of signal/noise ratios finding for anomalies of natural processes, the predictability problem is solved by analyses of temporal data sets of related measurements for key regions and their background within contextually coherent structures of natural targets and between particular boundaries of the structures.

The Formation of Topographic Maps That Maximize the Average Mutual Information of the Output Responses to Noiseless Input Signals

This article introduces an extremely simple and local learning rule for to pographic map formation. The rule, called the maximum entropy learning rule (MER), maximizes the unconditional entropy of the map's output for any type of input distribution. The aim of this article is to show that MER is a viable strategy for building topographic maps that maximize the average mutual information of the output responses to noiseless input signals when only input noise and noise-added input signals are available.

Topographic map formation by maximizing unconditional entropy: a plausible strategy for "online" unsupervised competitive learning and nonparametric density estimation

An unsupervised competitive learning rule, called the vectorial boundary adaptation rule (VBAR), is introduced for topographic map formation. Since VBAR is aimed at producing an equiprobable quantization of the input space, it yields a nonparametric model of the input probability density function. Furthermore, since equiprobable quantization is equivalent to unconditional entropy maximization, we argue that this is a plausible strategy for maximizing mutual information (Shannon information rate) in the case of "online" learning. We use mutual information as a tool for comparing the performance of our rule with Kohonen's self-organizing (feature) map algorithm.

Information, Asset Prices, and the Volume of Trade

ABSTRACTA dynamic equilibrium model is constructed in which agents with access to different information sets participate in the capital market. Agents must use the equilibrium price of capital to make optimal forecasts of the return to holding capital. Examples show that the volume of trade, as well as the price of capital, can be highly correlated with a measure of the information content of prices. This measure of information is the difference between the unconditional entropy of the dividend and the entropy of the dividend conditional on observing the price of capital.

The entropy of the maximum entropy distribution

The entropy of the univariate maximum entropy (ME) distribution is expressed in terms of order statistics. In the multivariate ME distribution, the expected mutual information equals a positive constant independent of the sample values, and the excess of the unconditional entropy of any variate over its average conditional entropy (given all other variates) equals the logarithm of the sample size.

Die merkmalanalyse für nominaldaten — eine methode zur analyse von qualitativen geographischen daten basierend auf einem informationstheoretischen modell

Mathematical analysis is not restricted any more to quantitative attributes. Methods have also been developed for the analysis of qualitative attributes, called “characters”. The present paper Intents to demonstrate that by introducing the so called “Character Analysis” to Geographers. Qualitative attributes (such as land use, language, religion, rock types, climate) can be recorded on nominal scales. The entropy-concept in information theory allows to assign a numerical value to each character (based on the probabilities of its states) which indicates the amount of information (or the uncertainty) of the character. The pairwise comparison of the conditional and the unconditional entropy and of the redundancy (information common in pairs of characters) results in a measure of similarity between characters. This method may be used as well for an evaluation of quality in a classification (or regionalization) or as a predictive model. An extension is the canonical character analysis, which allows for the comparison of two groups of characters. A thematical map is the classical geographical media of transmission of qualitative information. Therefore, in the worked example, 6 thematical maps from the atlas of Kenya are compared. This is certainly Just one among many possible and useful applications in Geography which has always ciaimed to be particularly interested in qualitative attributes of man and earth.