Application Of Information Theory Research Articles

Information theory entering soils and tissues.

For a long time, application of information theory for characterizing biological signaling fidelity was limited to "telegraph-like" situations, copying the classical scenario for which it was developed. A study published in this issue of Cell Systems applies this powerful framework to several distinct cases of dynamic signal sensing in complex geometries.

Measuring Creativity in the Fermi Problem, a Type of Mathematical Modelling, Applying Information Theory

Many methods of measuring creativity have been studied - mainly in psychology. In recent years, there have been attempts to incorporate such creativity into mathematical modelling, a topic handled in mathematics education. Accordingly, some studies have been trying to assess creativity in it. However, there have been no clear criteria or formulas that can be used for any problem, since a rubric has been created for each problem and evaluated individually. In the present study, to measure creativity in the Fermi problem, a type of mathematical modelling, a formula that applies information theory used in information science is proposed and examined using Structural Equation Modeling (SEM). A survey of Japanese junior high school students (n = 364) was conducted and analyzed, and the results show that the model using the formula proposed in the present study is a good fit. In addition, a moderate positive correlation (r = .41, p < .01 ) is found between creativity in the Fermi problem and creativity in psychology measured by the Test for Creative Thinking-Drawing Production (TCT-DP).

Synthetic gene networks recapitulate dynamic signal decoding and differential gene expression.

Cells live in constantly changing environments and employ dynamic signaling pathways to transduce information about the signals they encounter. However, the mechanisms by which dynamic signals are decoded into appropriate gene expression patterns remain poorly understood. Here, we devise networked optogenetic pathways that achieve dynamic signal processing functions that recapitulate cellular information processing. Exploiting light-responsive transcriptional regulators with differing response kinetics, we build a falling edge pulse detector and show that this circuit can be employed to demultiplex dynamically encoded signals. We combine this demultiplexer with dCas9-based gene networks to construct pulsatile signal filters and decoders. Applying information theory, we show that dynamic multiplexing significantly increases the information transmission capacity from signal to gene expression state. Finally, we use dynamic multiplexing for precise multidimensional regulation of a heterologous metabolic pathway. Our results elucidate design principles of dynamic information processing and provide original synthetic systems capable of decoding complex signals for biotechnological applications.

Surprisal analysis of diffusion processes

We report a preliminary step in the application of information theory motivated surprisal analysis to irreversible processes but having linear equations of motion. Here we aim to describe the dynamics of varied Smoluchowski diffusion processes. In surprisal analysis, the probability density over the sample space is written as an exponential function of linear combination of observables that pose as constraints to the density evolution. The coefficients of this expansion can be interpreted as Lagrange multipliers that arise in the maximum entropy formalism. Known solutions of diffusion processes can be reproduced exactly using such a description. An exponent that is a linear combination of observables can also provide a close approximation for the dominant behavior of the solution of some other interesting examples for which there are no known exact results.

Meaningful Secret Image Sharing with Saliency Detection.

Secret image sharing (SIS), as one of the applications of information theory in information security protection, has been widely used in many areas, such as blockchain, identity authentication and distributed cloud storage. In traditional secret image sharing schemes, noise-like shadows introduce difficulties into shadow management and increase the risk of attacks. Meaningful secret image sharing is thus proposed to solve these problems. Previous meaningful SIS schemes have employed steganography to hide shares into cover images, and their covers are always binary images. These schemes usually include pixel expansion and low visual quality shadows. To improve the shadow quality, we design a meaningful secret image sharing scheme with saliency detection. Saliency detection is used to determine the salient regions of cover images. In our proposed scheme, we improve the quality of salient regions that are sensitive to the human vision system. In this way, we obtain meaningful shadows with better visual quality. Experiment results and comparisons demonstrate the effectiveness of our proposed scheme.

Improved bounds for the Jensen gap with applications in information theory

Improved bounds for the Jensen gap with applications in information theory

COVID‐19 Lockdown Unravels the Complex Interplay between Environmental Conditions and Human Activity

During the COVID‐19 epidemic, draconian countermeasures forbidding nonessential human activities have been adopted in several countries worldwide, providing an unprecedented setup for testing and quantifying the current impact of humankind on climate and for driving potential sustainability policies in the postpandemic era from a perspective of complex systems. In this study, we consider heterogeneous sources of environmental and human activity observables, considered as components of a complex socioenvironmental system, and apply information theory, network science, and Bayesian inference to analyze their structural relations and nonlinear dynamics between January 2019 and August 2020 in northern Italy, i.e., before, during, and after the national lockdown. The topological structure of a complex system strongly impacts its collective behavior; therefore, mapping this structure is essential to fully understand the functions of the system as a whole and its fragility to unexpected disruptions or shocks. To this aim, we unravel the causal relationships between the 16 environmental conditions and human activity variables, mapping the backbone of the complex interplay between intervening physical observables—such as NO2 emissions, energy consumption, intervening climate variables, and different flavors of human mobility flows—to a causal network model. To identify a tipping point during the period of observation, denoting the presence of a regime shift between distinct network states (i.e., before and during the shock), we introduce a novel information‐theoretic method based on statistical divergence widely used in statistical physics. We find that despite a measurable decrease in NO2 concentration, due to an overall decrease in human activities, locking down a region as a climate change mitigation is an insufficient remedy to reduce emissions. Our results provide a functional characterization of socioenvironmental interdependent systems, and our analytical framework can be used, more generally, to characterize environmental changes and their interdependencies using statistical physics.

Cybernetics, Listening, and Sound-Studio Phenomenotechnique in Abraham Moles’s Théorie de l’information et perception esthétique (1958)

In his Théorie de l’information et perception esthétique (1958), the sociologist of culture Abraham Moles (1920–92) set out to demonstrate the applicability of information theory—a mathematical linchpin of cybernetics—to the arts more generally. Moles drew on classical psychophysics, Gestalt psychology, more modern behavioral psychology, and contemporary psychoacoustic research to advocate a cybernetic model of the perception and creation of art. Moles repeatedly returned to musical examples therein to make his case, leveraging his dual expertise in philosophy and electroacoustics, drawing on formative experiences with Pierre Schaeffer in Paris and Hermann Scherchen at his Gravesano studio. Moles’s interdisciplinary text found many attentive readers across Europe and, following an English translation by the precocious Joel E. Cohen (1966), the Anglophone academic world, but it was valued more as an inspiration for the burgeoning area of “information aesthetics” than as a source of hard scientific evidence. Drawing lightly on positions in the history and philosophy of science articulated by Gaston Bachelard (who supervised Moles’s second PhD, in philosophy) and Hans-Jörg Rheinberger suggests a change of emphasis away from its apparent scientific infelicities and toward Moles’s use of sound-studio technique, which is described with reference to the technologies available to Moles in the years leading up to the publication of the Théorie. Moles manipulated and processed sound recordings—filtering, clipping, and reversing them—in his attempts to empirically estimate the relative proportions of semantic and aesthetic information in speech and music. Moles’s text, when understood in tandem with the traces of his practical experiments in the sound studio, appears as an influential and occasionally prescient exposition of the many possible applications of the principles of information theory to the production, perception, and consumption of sound culture that makes ready use of the latest technical innovations in the media environment of its time.

Decoding of facial muscle-brain relation by information-based analysis of electromyogram (EMG) and electroencephalogram (EEG) signals

To understand the connection among the brain and facial muscles, we decoded the correlation of facial muscles and brain activities. We applied information theory (Shannon entropy) to EEG and EMG signals recorded from 12 subjects (7 M, 5 F) during rest and stimulation using three odors (pineapple, banana, vanilla). The results indicated that a larger variation in the molecular complexity of odors causes a larger variation in the information of EMG and EEG signals. Therefore, facial muscles’ activity is strongly coupled (r = −0.8399) to the brain’s activity. Our analysis method could be further employed to examine the relationship between the brain and other organs by comparing other physiological signals with the brain signals.

A new approach for the derivation of bounds for the Jensen difference

There are many useful applications of Jensen's inequality in several fields of science, and due to this reason, a lot of results are devoted to this inequality in the literature. The main theme of this article is to present a new method of finding estimates of the Jensen difference for differentiable functions. By applying definition of convex function, and integral Jensen's inequality for concave function in the identity pertaining the Jensen difference, we derive bounds for the Jensen difference. We present integral version of the bounds in Riemann sense as well. The sharpness of the proposed bounds through examples are discussed, and we conclude that the proposed bounds are better than some existing bounds even with weaker conditions. Also, we present some new variants of the Hermite–Hadamard and Hölder inequalities and some new inequalities for geometric, quasi‐arithmetic, and power means. Finally, we give some applications in information theory.

Reliable Recurrence Algorithm for High-Order Krawtchouk Polynomials.

Krawtchouk polynomials (KPs) and their moments are promising techniques for applications of information theory, coding theory, and signal processing. This is due to the special capabilities of KPs in feature extraction and classification processes. The main challenge in existing KPs recurrence algorithms is that of numerical errors, which occur during the computation of the coefficients in large polynomial sizes, particularly when the KP parameter (p) values deviate away from 0.5 to 0 and 1. To this end, this paper proposes a new recurrence relation in order to compute the coefficients of KPs in high orders. In particular, this paper discusses the development of a new algorithm and presents a new mathematical model for computing the initial value of the KP parameter. In addition, a new diagonal recurrence relation is introduced and used in the proposed algorithm. The diagonal recurrence algorithm was derived from the existing n direction and x direction recurrence algorithms. The diagonal and existing recurrence algorithms were subsequently exploited to compute the KP coefficients. First, the KP coefficients were computed for one partition after dividing the KP plane into four. To compute the KP coefficients in the other partitions, the symmetry relations were exploited. The performance evaluation of the proposed recurrence algorithm was determined through different comparisons which were carried out in state-of-the-art works in terms of reconstruction error, polynomial size, and computation cost. The obtained results indicate that the proposed algorithm is reliable and computes lesser coefficients when compared to the existing algorithms across wide ranges of parameter values of p and polynomial sizes N. The results also show that the improvement ratio of the computed coefficients ranges from 18.64% to 81.55% in comparison to the existing algorithms. Besides this, the proposed algorithm can generate polynomials of an order ∼8.5 times larger than those generated using state-of-the-art algorithms.

Least kth-Order and Rényi Generative Adversarial Networks.

We investigate the use of parameterized families of information-theoretic measures to generalize the loss functions of generative adversarial networks (GANs) with the objective of improving performance. A new generator loss function, least kth-order GAN (LkGAN), is introduced, generalizing the least squares GANs (LSGANs) by using a kth-order absolute error distortion measure with k≥1 (which recovers the LSGAN loss function when k=2). It is shown that minimizing this generalized loss function under an (unconstrained) optimal discriminator is equivalent to minimizing the kth-order Pearson-Vajda divergence. Another novel GAN generator loss function is next proposed in terms of Rényi cross-entropy functionals with order α>0, α≠1. It is demonstrated that this Rényi-centric generalized loss function, which provably reduces to the original GAN loss function as α→1, preserves the equilibrium point satisfied by the original GAN based on the Jensen-Rényi divergence, a natural extension of the Jensen-Shannon divergence. Experimental results indicate that the proposed loss functions, applied to the MNIST and CelebA data sets, under both DCGAN and StyleGAN architectures, confer performance benefits by virtue of the extra degrees of freedom provided by the parameters k and α, respectively. More specifically, experiments show improvements with regard to the quality of the generated images as measured by the Fréchet inception distance score and training stability. While it was applied to GANs in this study, the proposed approach is generic and can be used in other applications of information theory to deep learning, for example, the issues of fairness or privacy in artificial intelligence.

Applications of Information Theory in Rock Engineering

Rock engineering relies heavily on empirical systems to identify significant parameters influencing rock mass behaviour. The empirical and inductive nature of rock engineering design is such that it is not possible to eliminate uncertainty. One way of managing uncertainty during the design process is by collecting good quality data in a standardized and objective manner. However, difficulties arise when defining and determining what constitutes good quality data. We believe that information theory and the concept of Shannon’s entropy could be effectively used to better audit rock engineering data. This paper builds on established concepts by expanding and refining the application of information theory to rock mass classification systems, specifically the rock mass rating and the Q-system. One of the objectives is to provide and showcase a method whereby information auditing is used to flag uncertain (or poor quality) data. In the future it is not difficult to envision data collection processes that include improved core logging and data processing where imaging technologies are coupled with machine learning processing capability. Such an approach requires more quantitative and objective rock mass descriptions; in this context it easy to appreciate the role that information theory might have in the future in rock engineering.

Minimax Rates of ℓp-Losses for High-Dimensional Linear Errors-in-Variables Models over ℓq-Balls.

In this paper, the high-dimensional linear regression model is considered, where the covariates are measured with additive noise. Different from most of the other methods, which are based on the assumption that the true covariates are fully obtained, results in this paper only require that the corrupted covariate matrix is observed. Then, by the application of information theory, the minimax rates of convergence for estimation are investigated in terms of the -losses under the general sparsity assumption on the underlying regression parameter and some regularity conditions on the observed covariate matrix. The established lower and upper bounds on minimax risks agree up to constant factors when , which together provide the information-theoretic limits of estimating a sparse vector in the high-dimensional linear errors-in-variables model. An estimator for the underlying parameter is also proposed and shown to be minimax optimal in the -loss.

Bottom Communities and Abiotic Factors: Analysis of Statistical Relationship Using the Instability Index and Virtual Species Distribution

Statistical procedures for quantifying the relationships between the community structure and abiotic variables start with selecting a minimum set of uncorrelated environmental factors that determine the ecological conditions essential for each of the species. This is especially important when constructing models of spatial distribution of species which are key to ecology of communities and conservation of nature. The aim of the study is to explore whether some applications of information theory can be used to rank environmental factors with respect to their contribution to the formation of the ecological structure of aquatic communities. We consider the applicability of the instability index, which is a special case of the Kullback-Leibler entropy divergence and reflects the information gain from the displacement of a particular realization of a random variable relative to its mean value. Using of instability indices allows to reduce multidimensional data sets on species structure of communities and abiotic factors to lower dimension sets of commensurate standardized variables and to explore the relationships between the latter. The initial data we used were the results of the long-term (1990–2019) hydrobiological survey of benthic communities in small and medium-sized rivers in the Middle and Lower Volga regions. We consider the indices of instability calculated for each of 147 taxa of macrozoobenthos and 8 geophysical and hydrochemical indicators. Based on these data, we constructed random forest regression models and calculated potential weights of environmental factors that determine ecological preferences of species. The most significant explanatory variables were used to construct distribution maps of «virtual species», which were compared with the corresponding empirical data. A habitat suitability map of chironomids (Diptera, Chironomidae), the Prodiamesinae subfamily, is presented. Instability indices can be effectively used for exploratory analysis of various ecosystems, e. g. ranking habitats according to the degree of environmental instability and / or species associations, selecting the most informative abiotic variables that determine the population density of the taxa, etc.

Quantification of the Direct Solar Impact on Some Components of the Hydro-Climatic System.

This study addresses the causal links between external factors and the main hydro-climatic variables by using a chain of methods to unravel the complexity of the direct sun–climate link. There is a gap in the literature on the description of a complete chain in addressing the structures of direct causal links of solar activity on terrestrial variables. This is why the present study uses the extensive facilities of the application of information theory in view of recent advances in different fields. Additionally, by other methods (e.g., neural networks) we first tested the existent non-linear links of solar–terrestrial influences on the hydro-climate system. The results related to the solar impact on terrestrial phenomena are promising, which is discriminant in the space-time domain. The implications prove robust for determining the causal measure of climate variables under direct solar impact, which makes it easier to consider solar activity in climate models by appropriate parametrizations. This study found that hydro-climatic variables are sensitive to solar impact only for certain frequencies (periods) and have a coherence with the Solar Flux only for some lags of the Solar Flux (in advance).

Uniform Treatment of Jensen’s Inequality by Montgomery Identity

We generalize Jensen’s integral inequality for real Stieltjes measure by using Montgomery identity under the effect of n − convex functions; also, we give different versions of Jensen’s discrete inequality along with its converses for real weights. As an application, we give generalized variants of Hermite–Hadamard inequality. Montgomery identity has a great importance as many inequalities can be obtained from Montgomery identity in q − calculus and fractional integrals. Also, we give applications in information theory for our obtained results, especially for Zipf and Hybrid Zipf–Mandelbrot entropies.

Information, Thermodynamics and Life: A Narrative Review

Information is probably one of the most difficult physical quantities to comprehend. This applies not only to the very definition of information, but also to the physical entity of information, meaning how can it be quantified and measured. In recent years, information theory and its function in systems has been an intense field of study, due to the large increase of available information technology, where the notion of bit dominated the information discipline. Information theory also expanded from the “simple” “bit” to the quantal “qubit”, which added more variables for consideration. One of the main applications of information theory could be considered the field of “autonomy”, which is the main characteristic of living organisms in nature since they all have self-sustainability, motion and self-protection. These traits, along with the ability to be aware of existence, make it difficult and complex to simulate in artificial constructs. There are many approaches to the concept of simulating autonomous behavior, yet there is no conclusive approach to a definite solution to this problem. Recent experimental results have shown that the interaction between machines and neural cells is possible and it consists of a significant tool for the study of complex systems. The present work tries to review the question on the interactions between information and life. It attempts to build a connection between information and thermodynamics in terms of energy consumption and work production, as well as present some possible applications of these physical quantities.

Novel Ocean Futures: Anticipating Great Change in the World's Marine Protected Areas

Climate change has far-reaching implications for the continued utility of marine protected areas (MPAs) as the ocean's biogeochemistry is altered. We employed an application of information theory to measure the departure from historical conditions of six key marine biogeochemical variables for all parts of the ocean and the largest MPAs. We analyzed a multi-model ensemble (CMIP6) under three climate change scenarios and found that between 76-97% of the largest MPAs will exhibit significant departures from current conditions in their biogeochemical profiles, with most novel conditions emerging as soon as the 2030s (pH) and most others by 2060. These results suggest that many of the largest MPAs will contain environmental conditions unlike those currently within their boundaries, and, given the likelihood of any of these climate futures unfolding, present-day management may need to consider alterations to current and future MPA planning.

ART AS A STUDY OF AESTHETICALLY INFORMATION IMAGE

It is proved that aesthetic information is not identical with semantic or scientific information. Semantic information should be understood as new knowledge (message, experience in the process of aesthetic perception) about the object. It can be expressed by objective methods in a certain system of knowledge. In contrast, aesthetic information is new about the artist who conveys certain semantic information. Aesthetic information expresses the artist's relationship to the object he is reproducing. It is considered that cognition is not only a description of an object and a process of transmission and processing of information. Cognition is also the application of methods of thinking and logical operations to identify, reflect an object and create an "image of reality" in the human mind. The study of their detection in science is the subject of the logic of scientific knowledge. However, human use of methods of thinking and logical operations in science and art are different. The logic of the aesthetic relation is still a great mystery, although in all the processes that take place in art, one can see a kind of discovery of the methods of logical thinking and mediation. Thus, art as a study takes place within its individual types and genres, as happened, for example, with abstractionism in painting and music, which for almost a century experimented with the influence of various mixtures and relationships of color and sound on human perception; moreover, the Impressionist artists themselves often stated that the purpose of their work was to influence people through art and that the world around them was nothing but a combination of sounds, colors, light lines and shadows. Art as research can be manifested clearly in any single process that takes place in art, but in a process that determines the work of an artist. Leonardo la Vinci had a holistic nature, but he often became a scientist only when he conducted his art studies with research accuracy and scrupulousness. In the latter case, art is especially clearly transformed into its opposite - into science. The artist becomes a research scientist. He seeks to reproduce and study the process in art in more detail, and on the way to this goal he slowly deviates from the methods of artistic research, replacing them with the means and methods of scientific research. The article investigates an attempt to apply information theory to the study of art as cognition. The division of information into semantic and aesthetic is applied. The research function of art as its special side is revealed, and aesthetic cognition in this case is a part of aesthetic relation, where art as cognition is only a separate side of art as a kind of human activity.