Application Of Information Theory Research Articles

The nature of information and the implication for making predictions about soil properties using soil maps

Information theory is one method to evaluate how soil data can be utilised to estimate the amount of information produced by grouping or classifying the soil into classes or spatial polygons. Information theory provides the means of making “information” a measurable quantity. This paper explores how information theory can be applied to an existing, conventional soil map for the Lower Macquarie alluvial plain in New South Wales. It uses stepwise interpretation tables to estimate information levels for pH, salinity, ESP and shrink swell potential. An estimate is made of the amount of information produced for each of the soil profile classes (SPCs) and the soil map units. The amount of information produced varied between soil properties and for different soil depths. It also varied between the SPCs and soil map units. The soil map identified high amounts of available information for some soil properties, particularly in the surface soils for salinity and ESP. Moderate levels of information were generally available for pH. For other soil properties, only low amounts of information were available, especially for salinity and shrink swell potential in the deep subsoils. The overall estimate indicated that the soil map provided 50% of the total potential available information. The amount of available information produced for the soil properties was shown to be highly correlated to the standard deviation of the soil properties. The application of information theory and the use of stepwise interpretation tables was shown to be a useful method of evaluating the amount of information produced by a soil map for a range of soil properties.

The Application of Information Theory to Interpret Shore Platform Erosion Rates

Advancements in information physics have recently introduced the application of information theory to investigate physical systems. The behaviour of erosion at the granular scale is to date still a complex system to unpack, and therefore geomorphology research requires novel approaches to better inform the interpretation of temporal and spatial erosion patterns at different scales. This paper applies information theory concepts to re-evaluate erosional data that were measured on limestone surfaces of two shore platforms in Malta with a traversing micro-erosion meter (TMEM). By representing erosion rates through their information content using a Box-Cox style transformation of the raw data (application of an inverse normal distribution function to fractionally ranked data), it is possible to identify points and measurement periods that contribute to a disproportionately large share of unexpected erosion rates that could provide more insight into the causes of erosion rates. Despite the variations in the information content from erosion rates at individual measurement points, most points consistently contribute to a similar amount of information. These findings illuminate the importance of considering the informational value of erosion data to further understand the underlying physical processes and potentially improve predictive models.

Optimising the acquisition conditions of high information quality low-field NMR signals based on a cutting-edge approach applying information theory and Taguchi's experimental designs – Virgin olive oil as an application example

BackgroundDeveloping a new spectrometric analytical method based on a fingerprinting approach requires optimization of the experimental stage, particularly with novel instruments like benchtop low-field NMR spectrometers. To ensure high-quality LF-NMR spectra before developing the multivariate model, an experimental design to optimize instrument conditions is essential. However, difficult-to-control factors may be critical for optimization. Taguchi methodology addresses these factors to obtain a system robust to variation. This study uses the Taguchi methodology to optimize instrument settings for acquiring high-quality 1H and 13C LF-NMR signals in a short time from virgin olive oil (VOO). ResultsTwo experimental trials (for 1H and 13C signals, respectively) were carried out and analysed to find an optimal and robust combination of instrument settings against changes in two difficult-to-control factors: ambient temperature and small deviations of the NMR tube volume (700 ± 50 μL). The responses to be optimised, run time and spectral information quality, were analysed separately and jointly, as some factors showed opposite behaviour in the effect on the responses. Multiple response analysis based on suitable desirability functions yielded a combination of factors resulting in desirability values above 0.8 for 1H LF-NMR signals and almost 1.0 for 13C LF-NMR signals.In addition, a novel approach to assess the information quality of an analytical signal was proposed, addressing a major challenge in analytical chemistry. By applying information theory and calculating information entropy, this approach demonstrated its potential for selecting the highest quality (i.e. most informative) analytical signals. SignificanceThe acquisition instrument conditions of LF-NMR were successfully optimised using Taguchi methodology to acquire highly informative 1H and 13C spectra in a minimum run time. The importance lies in the future development of non-targeted analytical applications for VOO quality control. In addition, the innovative use of information entropy to a priori assess the signal quality represents a significant advance and proposes a solution to a long-standing challenge in analytical chemistry.

Wavelet-based information theory in quantitative assessment of AFM images’ quality

The quantitative assessment of the image quality produced by atomic force microscopy (AFM) is an ongoing and challenging task. In our study, we demonstrate Shannon’s application of information theory for measuring image quality. Specifically, we propose quantifying the loss of image information due to the various distortion processes by exploring the relationship between image information based on the information channel capacity (ICC), spectral image representation, and visual quality. Since the ideal image is unavailable, the power and noise spectrum, the critical input information for the image quality evaluation, must be robustly estimated in the proposed method. The classical, most popular Welch method for spectral estimation uses an average of several windowed periodograms and can produce biased spectrum estimates. Therefore, in our work, we discuss an alternative technique based on the wavelet transform that can be applied to solve this challenging problem, specifically in the case of noisy, uncertain AFM measurements. Finally, we validate the performance of the enhanced ICC-wavelet-based algorithm with noisy measurement AFM data.

New bounds of Popoviciu's difference via weighted Hadamard type inequalities with applications in information theory

We generalize Popoviciu's inequality utilizing the theory of 4‐convex function under the effect of new Green functions. We formulate result for power means and quasi means. Also, we give applications in information theory by giving new estimations of generalized Csiszár divergence, Rényi divergence, Shannon entropy, Kullback–Leibler divergence, divergence, Zipf–Mandelbrot entropy, and Hybrid Zipf–Mandelbrot entropy.

New Bounds of Cyclic Jensen’s Differences via Weighted Hadamard Inequalities with Applications

ABSTRACT We generalize cyclic Jensen’s inequality utilizing the theory of 4-convex function under the effect of introduced Green functions. We formulate result for power means and quasi means. Also we give applications in information theory by giving new estimations of generalized Csiszár divergence, Rényi-divergence, Shannon-entropy, Kullback-Leibler divergence and χ 2-divergence.

Information-Theoretic Analysis of a Model of CAR-4-1BB-Mediated NFκB Activation

Systems biology utilizes computational approaches to examine an array of biological processes, such as cell signaling, metabolomics and pharmacology. This includes mathematical modeling of CAR T cells, a modality of cancer therapy by which genetically engineered immune cells recognize and combat a cancerous target. While successful against hematologic malignancies, CAR T cells have shown limited success against other cancer types. Thus, more research is needed to understand their mechanisms of action and leverage their full potential. In our work, we set out to apply information theory on a mathematical model of NFκB signaling initiated by the CAR following antigen encounter. First, we estimated channel capacity for CAR-4-1BB-mediated NFκB signal transduction. Next, we evaluated the pathway’s ability to distinguish contrasting “low” and “high” antigen concentration levels, depending on the amount of variability in protein concentrations. Finally, we assessed the fidelity by which NFκB activation reflects the encountered antigen concentration, depending on the prevalence of antigen-positive targets in tumor population. We found that in most scenarios, fold change in the nuclear concentration of NFκB carries a higher channel capacity for the pathway than NFκB’s absolute response. Additionally, we found that most errors in transducing the antigen signal through the pathway skew towards underestimating the concentration of encountered antigen. Finally, we found that disabling IKKβ deactivation could increase signaling fidelity against targets with antigen-negative cells. Our information-theoretic analysis of signal transduction can provide novel perspectives on biological signaling, as well as enable a more informed path to cell engineering.

Distribution of Patterns of Constrained Length in Binary Sequences

On a finite sequence of binary (0-1) trials we define a random variable enumerating patterns of length subject to certain constraints. For sequences of independent and identically distributed binary trials exact probability mass functions are established in closed forms by means of combinatorial analysis. An explicit expression of the mean value of this random variable is obtained. The results associated with the probability mass functions are extended on sequences of exchangeable binary trials. An application in Information theory concerning counting of a class of run-length-limited binary sequences is provided as a direct byproduct of our study. Illustrative numerical examples exemplify further the results.

The Kelly Criterion: Optimizing Decision-Making in Risk Management

The Kelly capital growth investment criterion, or Kelly criterion, defines the fraction of wealth to invest in a favorable investment opportunity such that the exponential growth rate is maximized. Maximizing the exponential growth rate is equivalent to maximizing logarithmic utility.[1] It all began in the mid-20th century when John L. Kelly Jr., a researcher at Bell Labs, developed the criterion as a solution to a problem related to the efficient transmission of information in telecommunications. Kelly introduced the concept of the Kelly Criterion within his paper “A New Interpretation of Information Rate,” which he published in his early years. However, he did not refer to the Kelly Criterion by its now- known name. The Kelly Criterion initially gained recognition in academic and mathematical circles, primarily for its applications in information theory. Then, professional gamblers and investors started using the Kelly Criterion to manage their bankrolls and make more informed betting decisions. The financial industry also embraced the Kelly Criterion as an alternative approach to portfolio management and investment strategies. The Kelly Criterion gained further attention in investment circles, particularly in hedge funds and wealth management.

An information theory approach to identifying signs of life on transiting planets

ABSTRACT Can information theory provide insights into whether exoplanets are habitable? Here, we apply information theory to a range of simulated exoplanet transmission spectra as a diagnostic tool to search for potential signatures of life on Earth-analogue planets. We test the algorithms on three epochs of evolution for Earth-like planets orbiting a range of host stars. The JWST and upcoming ground- and space-based missions promise to achieve sufficient high-resolution data that information theory can be applied to assess habitability. This approach provides a framework and a tool for observers to assess whether an exoplanet shows signs of habitability.

Derivation of Bounds for Majorization Differences by a Novel Method and Its Applications in Information Theory

In the recent era of research developments, mathematical inequalities and their applications perform a very consequential role in different aspects, and they provide an engaging area for research activities. In this paper, we propose a new approach for the improvement of the classical majorization inequality and its weighted versions in a discrete sense. The proposed improvements give several estimates for the majorization differences. Some earlier improvements of the Jensen and Slater inequalities are deduced as direct consequences of the obtained results. We also discuss the conditions under which the main results give better estimates for the majorization differences. Applications of the acquired results are also presented in information theory.

New cyclic Jensen type estimations VIA weighted Hadamard inequalities under the effect of Fink's identity

We take into account differences resulting from the discrete and integral cyclic Jensen's inequalities and provide upper and lower bounds using weighted Hermite-Hadamard inequalities with the support of Fink's identity. The theory of n-times differentiable convex functions is used to demonstrate this case. Our findings are applicable to any n ​≥ ​2, and we present specific examples to demonstrate the accuracy of the bounds found for unique situations. We also formulate results for power means and quasi-arithmetic means for higher order convexity. Last but not least, we provide applications in information theory by offering new uniform estimations of the generalized Csiszár, Rényi, Shannon, Kullback-Leibler, χ2 ​− ​divergence, Bhattacharyya, Zipf, and Hybrid Zipf-Mandelbrot entropies.

Feature subset selection for data and feature streams: a review

Real-world problems are commonly characterized by a high feature dimensionality, which hinders the modelling and descriptive analysis of the data. However, some of these data may be irrelevant or redundant for the learning process. Different approaches can be used to reduce this information, improving not only the speed of building models but also their performance and interpretability. In this review, we focus on feature subset selection (FSS) techniques, which select a subset of the original feature set without making any transformation on the attributes. Traditional batch FSS algorithms may not be adequate to efficiently handle large volumes of data, either because memory problems arise or data are received in a sequential manner. Thus, this article aims to survey the state of the art of incremental FSS algorithms, which can perform more efficiently under these circumstances. Different strategies are described, such as incrementally updating feature weights, applying information theory or using rough set-based FSS, as well as multiple supervised and unsupervised learning tasks where the application of FSS is interesting.

Linear Diophantine fuzzy sets over complex fuzzy information with applications in information theory

This article aims to link AG-groupoids and complex fuzzy set theory by constructing a method to rank complex fuzzy information, provide an alternative definition of CLDFS, and an algorithm for data encryption/decryption using AG-groupoids. Achieving these objectives would enable effective handling of complex and ambiguous data and a novel approach to data security with potential applications in various industries. The authors introduce new operations and laws to explore the structural properties of CLDFS, including applying the inverse discrete Fourier transform of a CLDFS to identify a reference signal and characterizing right regular ordered AG-groupoids in terms of CLDF-score right ideals. They also demonstrate an application of utilizing an AG-groupoid as a symmetric key for encryption/decryption and CLDF-score right ideals of an ordered AG-groupoid to select a suitable signal for system analysis.

Use of the entropy approach in water resource monitoring systems

Effective management of water resources is possible only with an effectively organized monitoring system. After the emergence and development of information theory, the concept of information entropy found its place in the field of the development of water monitoring systems. The purpose of this work is to review research related to the construction of water monitoring systems and networks that applied the entropy theory in the design process. Methodology. Entropy terms used in the construction of water monitoring systems are summarized. Recent applications of the entropy concept for water monitoring system designs classified by precipitation are reviewed; flow and water level; water quality; soil moisture and groundwater. The integrated method of designing multifactorial monitoring systems is also highlighted. Results. The review analyzes studies and their implementation in the design of water monitoring networks based on entropy. The use of various methods of information theory and their adaptation for use in the design of monitoring networks is demonstrated, with the goal of network design methods being the selection of stations that provide the most information for the monitoring network, while being independent of each other. Through extensive testing, information theory has proven to be a reliable tool for evaluating and designing an optimal water monitoring network. Scientific novelty. This review focuses on studies that have applied information theory or information entropy to construct monitoring networks and systems. Information theory was developed by Shannon in the middle of the last century to measure the information content of a data set and was subsequently applied to solving water resources problems. To date, there are no review studies regarding the design of water monitoring networks based on the concept that entropy will be able to characterize the information specific to the monitoring station or monitoring networks. The main goal is to have the maximum amount of information. Practical significance. The optimal design of the monitoring network can be built based on the specified design criteria; however, the practical application of a new optimal monitoring network is rarely evaluated in a hydrological or other model. It is also important to identify the benefits of entropy-based network design to convince decision-makers of the importance of entropy-based approaches. The optimal network can be subjective, based on the choices made during the entropy calculation and the design method chosen, especially when additional objective functions are considered in the design. This applies to the method chosen to construct the optimal monitoring network, whether it is found using an iterative method where one station is added at a time, or a collection of stations that are added simultaneously. Research has also shown that data length, catchment scale, and the order can affect optimal network design. when using discrete entropy, it was shown that the binning method affects the final network design. Therefore, when selecting options based on the intended application of the monitoring network, a clear understanding and further research is needed to provide recommendations specific to water monitoring networks. In particular, more work is needed on the spatial and temporal scaling of the entropy calculation data to provide robust recommendations for decision-makers.

Investigation of Dynamical Complexity in Swarm-Derived Geomagnetic Activity Indices Using Information Theory

In 2023, the ESA’s Swarm constellation mission celebrates 10 years in orbit, offering one of the best ever surveys of the topside ionosphere. Among its achievements, it has been recently demonstrated that Swarm data can be used to derive space-based geomagnetic activity indices, similar to the standard ground-based geomagnetic indices monitoring magnetic storm and magnetospheric substorm activity. Recently, many novel concepts originating in time series analysis based on information theory have been developed, partly motivated by specific research questions linked to various domains of geosciences, including space physics. Here, we apply information theory approaches (i.e., Hurst exponent and a variety of entropy measures) to analyze the Swarm-derived magnetic indices from 2015, a year that included three out of the four most intense magnetic storm events of the previous solar cycle, including the strongest storm of solar cycle 24. We show the applicability of information theory to study the dynamical complexity of the upper atmosphere, through highlighting the temporal transition from the quiet-time to the storm-time magnetosphere, which may prove significant for space weather studies. Our results suggest that the spaceborne indices have the capacity to capture the same dynamics and behaviors, with regards to their informational content, as traditionally used ground-based ones.

On Hierarchies of Balanced Sequences

Balanced sequences and balanced codes have attracted a lot of research in the last seventy years due to their diverse applications in information theory as well as other areas of computer science and engineering. There have been some methods to classify balanced sequences. This work suggests two new different hierarchies to classify these sequences. The first one is based on the largest <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell $ </tex-math></inline-formula> for which each <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell $ </tex-math></inline-formula> -tuple is contained the same amount of times in the sequence. This property is a generalization for the property required for de Bruijn sequences. The second hierarchy is based on the number of balanced derivatives of the sequence. Enumeration for each such family of sequences and efficient encoding and decoding algorithms are provided in this paper.

Several Jensen–Grüss Inequalities with Applications in Information Theory

Several Jensen–Grüss Inequalities with Applications in Information Theory

Thermodynamics of signal transduction systems and fluctuation theorem in a signal cascade

Biochemical chain reactions are signal transduction cascades that can transmit biological information about the intracellular environment. In this study, we modelled a chain reaction as a code string for applying information theory. Herein, we assumed that cell signal transduction selects a strategy to maximize the transduced signal per signal event duration. To investigate the same, we calculated the information transmission capacity of the reaction chain by maximizing the average entropy production rate per reaction time, indicating the idea of the entropy coding method. Moreover, we defined a signal cascade trajectory. Subsequently, we found that the logarithm of the forward and reverse transition ratio per reaction time is equal to the entropy production rate, which derives the form of the fluctuation theorem in signal transduction. Our findings suggest the application of information entropy theory for analysing signal transduction.

Analysis of the Relationship between Creativity in Fermi Problems Measured by Applying Information Theory, Creativity in Psychology, and Mathematical Creativity

Many educational institutions demand the development of creativity. However, it is still insufficient for encouraging creative work or thinking. One reason is the lack of simple tools to measure creativity in schools. This study focused on Fermi problems to solve the reason for this issue. Fermi problems have been suggested to be deeply related to creativity. However, few empirical studies have been conducted on their relationship. Therefore, this study conducted a survey and analyzed them by structural equation modeling. The results showed a moderate correlation between creativity in Fermi problems and creativity in psychology (r = 0.47, p &lt; 0.01). Additionally, it was shown that there is a strong correlation between creativity in Fermi problems and mathematical creativity (r = 0.76, p &lt; 0.01). Furthermore, regression analysis showed that creativity in Fermi problems is an important factor for measuring creativity in psychology and mathematical creativity.